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ABSTRACT 



This report is part of a multi-year project to examine the 
effects of standards -based assessment reform on schools and classrooms. 

During the past year, the project investigated school and classroom effects 
of assessment reforms in the state of Kentucky, a state that has been at the 
forefront of the standards-based assessment movement. This report presents 
the results of a series of surveys of the impact of the Kentucky 
Instructional Results Information System (KIRIS) on curriculum and 
instruction in mathematics in grades 5 and 8 and in writing in grades 4 and 
7. Teachers' self-reports (n=approximately 400) indicated substantial changes 
in classroom practices in response to the Kentucky Education Reform Act and 
KIRIS. KIRIS, in particular, has had a major influence on instructional 
behavior according to teachers. However, there were no consistent 
associations between reports of specific teaching practices or changes in 
practice during the past 3 years and biennial KIRIS gains. (Contains 2 
figures, 31 tables, and 15 references.) (Author/SLD) 
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Brian M. Stecher, Sheila Barron, Tessa Kaganoff, and Joy Goodwin 
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ABSTRACT 

This report is part of a multi-year project to examine the effects of standards-based 
assessment reform on schools and classrooms. During the past year, the project 
investigated school and classroom effects of assessment reforms in the state of 
Kentucky, a state that has been at the forefront of the standards-based assessment 
movement. This report presents the results of a series of surveys of the impact of the 
Kentucky Instructional Results Information System (KIRIS) on curriculum and 
instruction in mathematics in grades 5 and 8 and in writing in grades 4 and 7. Teachers' 
self-reports indicated substantial changes in classroom practices in response to KERA 
and KIRIS. KIRIS, in particular, has had a major influence on instructional behavior 
according to teachers. However, there were no consistent associations between reports of 
specific teaching practices or changes in practice during the past three years and 
biennial KIRIS gains. 



1 This project would not have been possible without encouragement and assistance from the staff 
from the Kentucky Department of Education and cooperation from teachers across the state. In 
particular, we want to acknowledge the support of Edward Reidy, Deputy Commissioner, and staff 
from the Kentucky Department of Education, including Brian Gong, Sue Rigney, Starr Lewis, Joanne 
Mosier, and Jonathan Dings. In addition, we received helpful suggestions from a number of Kentucky 
regional resource teachers, who were promised anonymity, and from hundreds of Kentucky 
classroom teachers, who took the time to complete our surveys. 

Our RAND colleagues Kathy Rosenblatt, Suzanne Perry, and Linda Daly deserve credit for 
coordinating the statewide survey effort, including production, distribution, monitoring, review, 
and data editing. 



BACKGROUND 



Focus of the Research 

In the past few years, there have been important changes in the nature of 
state assessment program. These changes include the introduction of more 
performance assessments (including open-response written measures, hands-on 
tasks, and cumulative portfolios), the alignment of assessments with state 
standards, and the use of assessment systems for school accountability purposes. 
Various reasons are cited for these changes. For example, proponents of 
performance assessments believe that “authentic" measures will lead to more 
effective instructional practices and will foster children's conceptual 
understanding (e.g.. Resnick & Resnick, 1992; Stiggins, 1991; Wiggens, 1989). The 
alignment of assessments with state standards is intended to promote improved 
classroom practice and to provide data for school improvement (Bond, 1994). 
The implementation of high-stakes accountability systems is supposed to 
increase attention to performance standards and student outcomes and create 
mechanisms to change ineffective practices. 

Most recent research on large-scale assessment has focused on the quality of 
scores and their appropriateness for accountability purposes. There has been very 
little research the effects of standards-based assessments on classroom practices. 
States and districts that change their assessment systems in one or more of these 
ways usually rely upon schools and teachers to translate the larger goals into 
practice. Little attention has been given to the processes through which schools 
implement and support assessment-based change and teachers adapt the 
classroom instructional environment to accommodate new curriculum 
standards and broadened achievement expectations. 

This is the first report from a multi-year project investigating the 
consequences of standards-based assessment reform at the school and classroom 
levels. The project is designed to address two broad questions: (1) What are the 
effects of recent statewide assessment reforms on school structures, classroom 
practices, and teachers' and students' understandings of assessment? (2) Which 
practices are associated with improvements in assessment results? Information 
about these processes is crucial to understanding why reform succeeds in some 
schools and falls short of expectations in others. During the past year, the project 
investigated teachers' classroom responses to assessment reforms in the state of 
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Kentucky. Kentucky was selected for study because it has been at the forefront of 
the standards-based assessment movement in this country. 

Standards-Based Educational Reform in Kentucky 

Kentucky implemented a new state assessment in 1991 as part of a broader 
school reform effort embodied in the Kentucky Education Reform Act (KERA). 
KERA sets learning standards for all students, and KIRIS — the Kentucky 
Instructional Reform Information System — measures students' progress toward 
meeting these standards (Kentucky Department of Education, 1994). 

The guiding principle behind KERA is that "all children can learn at high 
levels." To this end KERA delineates six learning goals, four of which address 
specific academic learning. These four goals include basic communication and 
mathematics skills, application of concepts and principles from academic subjects 
to real life situations, thinking and problem solving in school and life situations, 
and connecting and integrating knowledge across subjects. Each goal is supported 
by a list of detailed academic expectations for what students will know and be 
able to do. 

KERA also calls for dramatic changes in the way schools approach 
education. The state's philosophy is expressed in Transformations: Kentucky's 

Curriculum Framework, (Kentucky Department of Education, 1995). The 
document calls for "a philosophical change from traditional approaches to 
education" (Kentucky Department of Education, 1995b, p. 1). For Kentucky 
educators, this change is described by the phrase "standards-based education." A 
standards-based education emphasizes learning that matters, authenticity of 
contexts, and realistic demonstrations of performance (Kentucky Department of 
Education, 1995b, p. 6). 

The new assessment system, KIRIS, was designed to be consistent with the 
philosophy and content emphasis of KERA as well as with themes that 
characterize assessment reform nationally. First, KIRIS is performance-based in 
that it relies on constructed-response measures rather than multiple-choice 
items. 2 The assessment includes "on-demand," open-response questions in 
several subjects as well as yearlong portfolios in mathematics and writing. 
Second, KIRIS measures student achievement against specific standards that are 



2 Multiple-choice questions have been included in the assessment off and on over the years but they 
have not counted in the accountability index. 



set at high levels relative to current performance. Third, KIRIS is a "high stakes" 
assessment. An annual accountability index is computed for each school based 
on students' achievement scores and a set of non-cognitive measures, such as 
attendance and retention. Past performance is used as a baseline to establish 
improvement targets. For the purposes of rewards and sanctions, Kentucky's 
accountability system uses two-year averages rather than annual scores. If a 
school's overall two-year average score exceeds the target based on the previous 
biennium, the school receives a financial reward. If a school performs poorly for 
an extended period, the Department intervenes in the operation of the school, 
for example, by assigning a "distinguished educator" to consult with the school 
administrators. 3 

The Kentucky curriculum framework argues that a "complete 
transformation of the learning environment" will be required to insure that all 
children achieve at high levels (KDE, 1995b, p. 3). Districts have the responsibility 
for seeing that students achieve KERA's learning goals and academic 
expectations and for making whatever changes are necessary in curriculum and 
instruction to accomplish this objective. KDE believes that "success with all 
children will not change unless our practices change significantly" (KDE, 1995b, 

p. 10). 

What will these changes look like? The authors of Transformations draw 
clear distinctions between traditional education, which has not succeeded in 
Kentucky, and "standards-based education," which they believe holds the key to 
success. Table 1 summarizes eight ways in which "standards-based education" 
differs from the traditional approach. These broad statements can be translated 
into specific classroom practices, such as students working cooperatively to solve 
problems, inter-disciplinary lessons, and open-ended investigations. The 
practices encouraged by Kentucky are consistent with the reforms advocated in 
"Standards" documents prepared by the National Council of Teachers of English, 
the National Council of Teacher of Mathematics, and other professional 
organizations. The balance between such "standards-based" practices and more 



3 KDE has made a number of changes to KIRIS in response to research findings about the reliability 
of scoring and validity of scores for accountability purposes. For example, KIRIS originally 
included multi-disciplinary performance events, but these were dropped from the assessment 
because of the lack of a satisfactory procedure for linking assessments from year to year. More 
recently, the mathematics portfolios were placed "an hiatus" for two years while KDE reviewed 
their design and scoring criteria. 



traditional instructional activities serves as one measure of the effects of 
KERA/KIRIS and other reforms. 4 

Structure of the KIRIS Assessments 

Schools are held accountable for student performance in seven subject areas: 
reading, writing, mathematics, science, social studies, arts and humanities, and 
practical living /vocational studies. In the 1996-97 school year, testing was divided 
across grades to limit the burden on teachers and students in any single grade. At 
the elementary school level, students received on-demand assessments in 
reading, science and writing in grade four and in mathematics, social studies, arts 
and humanities, and practical living/ vocational studies in grade five. The same 
breakdown of subject areas was used for students in middle schools at grades 
seven and eight. At the high school level, eleventh-grade students completed 
on-demand assessments in all seven subject areas. In addition, teachers in 
fourth, seventh and twelfth grades had their students compile writing portfolios. 
Mathematics portfolios, which have been part of KIRIS in grades five, eight, and 
twelve, were officially on hiatus during the 1996-97 school year. This study did 
not explore the effects KIRIS at the high school level - 

Previous CRES ST/RAND Research in Kentucky 

RAND has been engaged in an ongoing program of research on the quality 
and effects of large-scale educational assessments (Koretz et al., 1994; Stecher & 
Mitchell, 1995). In 1994 RAND began to study Kentucky's assessment system. The 
first study consisted of surveys of fourth-grade teachers and eighth-grade 
mathematics teachers as well as school principals across Kentucky (Koretz et al., 
1996: These educators were asked about their opinions of KIRIS; changes they 
had made in instruction, assessment and school management; the methods they 
used to prepare students for KIRIS; and their implementation of the portfolio 
component of the assessment. In many ways, the current study is a continuation 
of this earlier effort, and the results of that research inform the present 
investigation. 



4 KDE staff pointed out that a teacher might be engaging in reform activities but not really 
adopting a standards-based approach. As they see it, a standards-based approach requires that 
activities be organized in a planned sequence or structure that is focused an the long-term 
accomplishment of a learning goal. The presence of reform activities is a necessary but not sufficient 
condition for education to be standards-based. Our surveys contain some question about organizing 
curriculum units, but focus mostly on instructional activities. 
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Table 1 



Comparison of Traditional and Standards-Based Education 

Traditional Approach Standards-Based Approach 



Calendar-Driven Instruction: School structured 
by calendar, e.g., age grouping, scheduling, etc. 

Constrained Opportunities: Limited instruc- 
tional strategies, physical environment, time 

Cumulative achievement: Work on discrete 
skills in predetermined time frames 

Competitive Learning: Individual environment 
with competition 

Comparative Evaluation: Based on relative 
performance of other students 

Curriculum Coverage: Exact knowledge 
dissemination within a predetermined 
timeframe 

Segmented Content: Discrete skills with few 
connections 

Curriculum Design: Focus on segmented coverage 



Standards-Driven Instruction: Success based on 
performance 

Expanded opportunities: Teaching time 
determined by learning and curriculum needs 

Culminating Achievement: Culminating 
achievement at the end of a learning cycle 

Cooperative Learning: Cooperative environment 
with self-directed challenges 

Criterion Evaluation: Based on set standards of 
quality 

Instructional Coaching: Finding instructional 
tools to enable students to demonstrate standards 
given appropriate time 

Connected Content: Integration within and 
between disciplines, connectedness 

Design Down: Focus on culminating performances 



Note. Adapted from Kentucky Department of Education (1995b), p. 9. 



With regard to classroom practices, Koretz et al. (1996) found that most 
principals and teachers felt KIRIS had more than a small positive effect on 
instruction in their schools. Principals reported focusing a great deal on 
encouraging teachers to improve instruction — both generally and to prepare 
students for KIRIS. Most principals also encouraged their teachers a great deal to 
focus instruction on skills and content likely to be on KIRIS, and nearly all 
principals reported a moderate or great increase in emphasis on this material. 
Teachers reported increasing the match between the content of their instruction 
and the content of the assessments. This included increased emphasis on 
problem solving, communicating mathematics, and writing. Teachers also 
increased their use of assessment formats^ther than -multiple-choice. 

Virtually all teachers agreed that KIRIS had caused teachers to de-emphasize 
or neglect untested material, although most principals did not concur that there 
had been a decrease in emphasis on material not tested by KIRIS in their schools. 



In particular, teachers reported less emphasis on the mechanics of writing and on 
mathematics computation and algorithms. 5 

METHODOLOGY 
Data Collection 

During 1996-97, we surveyed a representative sample of about 400 teachers 
from across the state of Kentucky. We also conducted case studies of a dozen 
exemplary teachers. Surveys were sent to elementary- and middle-school 
teachers in the KIRIS accountability grades for writing (fourth grade and seventh 
grade) and mathematics (fifth grade and eighth grade). We selected these subjects 
because of their importance in the school curriculum and because both were 
assessed using portfolios, one of the more innovative component of KIRIS. This 
report summarizes the results of the survey component of the project. 
Companion reports will describe the case study findings. 

Sampling 

Two stratification variables were used in drawing the sample: gain on the 
KIRIS accountability index in the subject of interest, and school size. Schools 
were placed into three equal strata (low, medium, and high) based on their gain 
in writing or mathematics during the second biennial cycle (1992-94 vs. 1994-96). 
Schools were also placed into two equal strata (small and large) based on school 
size. Schools with fewer than 20 students in the accountability grade were 
excluded from the sampling frame, as were schools with recent changes in their 
service areas. 

Within each stratum a random sample of schools was chosen. For each of 
the survey populations (fourth-grade writing, fifth-grade mathematics, seventh- 
grade writing, and eighth-grade mathematics) approximately 70 schools were 
selected. No school was chosen for more than one sample. Low- and high-gain 
schools were over-sampled (two low-gain and two high-gain schools were 
sampled for every one medium-gain school). The purpose of over-sampling was 
to increase the power for detecting differences in classroom practices between 



5 A recent study of fourth graders' on-demand writing found no change in writing mechanics 
(capitalization, punctuation, and subject/ verb agreement) from 1993 to 1996 (Hoffman, Koger, & 
Awbrey, 1997). 



low- and high-gain schools — one of the main comparisons of interest in this 
study. 

A letter was sent to the principal of each school at the beginning of 1997 
explaining the study and requesting the names of the instructors teaching the 
identified grade (and subject for middle schools). Principals were subsequently 
contacted by telephone to retrieve these names. Ninety-five percent of the 
principals in the sampled schools provided the requested information. 

The teachers were contacted by mail and asked to participate in the study. 
The contact letter explained the study and asked for their participation. Enclosed 
with the request were a letter from the Department of Education urging teacher 
cooperation, a copy of the survey to be completed, a return envelope, and a ten 
dollar gift certificate for purchasing books or other instructional materials. 
Teachers could keep the gift certificate regardless of whether they returned the 
survey. 

The overall teacher response rate was 70 percent. Table 2 contains the 
numbers of teachers who returned completed surveys in each of the four 
samples. 



Table 2 



Survey Sample and Response Rates 



Sample 


Schools 

Sampled 


Principals 

Providing 

Names 


Teachers 

Contacted 


Teachers 

Returning 

Survey 


Teacher 

Response 

Rate 


Fourth-Grade 

Writing 


71 


65 


177 


136 


77% 


Fifth-Grade 

Math 


66 


63 


125 


83 


66% 


Seventh-Grade 

Writing 


67 


66 


141 


95 


67% 


Eighth-Grade 


68 


65 


116 


77 


66% 


Math 




• - 


- - 







Survey Development 

The surveys addressed a broad range of issues related to classroom practices 
and the goals of KERA. Major themes included school and class organization. 



curriculum and instruction, classroom assessment, grading practices, and school 
support for change. In developing the surveys, we did an extensive review of 
instructional practice questionnaires used in other jurisdictions (e.g.. Porter & 
Smithson, 1995; Center on Organization and Restructuring of Schools, 1994). W e 
adopted or adapted some items from these sources and developed many new 
items specifically for this study. Most of the survey items focused on behaviors, 
but we also asked for teachers' opinions about a number of issues, including the 
KIRIS assessments and their impact on schools, classroom practices, and student 
performance. The surveys also contained questions related to teacher background 
and professional development. 

One of our purposes was to examine contrasts between practices that were 
consistent with Kentucky's view of standards-based education and practices that 
were consistent with traditional views of education (see Table 1). To this end, we 
developed questions that included practices that typified both approaches. For 
example, a question about mathematics teaching strategies permitted 
respondents to indicate the frequency with which they focused on computational 
skills and problem solving skills. 

It is important to note that Kentucky's views of standards-based education 
and traditional education are not mutually exclusive. Although many 
instructional practices are identified more closely with either a standards-based 
approach or a traditional approach, many other practices are compatible with 
both approaches. As a result, the items in our survey do not always align with 
only one approach. In this report, we have tried to focus on responses that most 
would agree are consistent with one approach or the other. However, we admit 
that this classification is not perfect, and we caution the reader to use his or her 
own judgment when interpreting the findings. 

Most of the survey questions were presented in a closed format. 
Respondents were asked to provide numerical answers or to select one option 
from a predetermined set of options (e.g., three-, four-, and five-point Likert 
scales, and yes /no questions).- Some-questions permitted respondents to write in 
other behaviors. For most questions about practice, teachers were asked about 
current behaviors (during the 1996-97 school year) and about changes during the 
past three-year period (which comprised the remainder of the accountability 
cycle). Only teachers with at least three years of experience answered questions 
about changes in practice. About 15 percent of the elementary teachers and about 



20 percent of the middle school teachers indicated that they could not answer 
these questions. 

Data Analysis 

We used weights to calculate descriptive statistics because the sampling plan 
included over-sampling of high-gain and low-gain schools. The weight assigned 
to each case was the product of the inverses of the probability that the school 
would be selected and probability that the sampled individuals would participate 
(complete the survey). 

Overall descriptive statistics were calculated separately for each grade and 
for teachers in high-gain and low-gain schools within each grade. We also 
combined grades to compute statistics by subject (e.g., the fourth- and seventh- 
grade responses were combined to calculate overall results for writing). When 
data were combined across grades, the two grades were weighted equally in the 
combined statistics. For the Likert questions, frequencies were computed. For 
questions requiring a numerical response, we calculated means, medians, and 
standard deviations. 

We tested the significance of the differences between responses for teachers 
in low- and high-gain schools based on second biennium gain scores using chi- 
squared tests and t-tests, as appropriate. The majority of statistical tests performed 
were chi-squared tests comparing two groups (teachers in low-gain schools and 
teachers in high-gain schools) and two levels of response (e.g., no /yes, low /high, 
or less frequently /more frequently). For this type of comparison and the smallest 
sample we obtained, a difference in the results for low- and high-gain teachers of 
anywhere from 16 percent to 30 percent would be needed for statistical 
significance at an alpha of .05, using no correction for multiple comparisons, and 
assuming a simple random sample. 6 Differences between mean scores in high- 
gain and low-gain schools were tested using t-tests. However, we do not 



6 We often surveyed more than one teacher.in a school, and ihus.it is likely ..that .there is at least a 
small clustering effect in the data. For this reason, simple random sample estimates provide only a 
guide for judging the significance of comparisons. To the extent that a clustering effect is present, 
accounting for it in significance testing would increase the differences needed to reach statistical 
significance. The reader should also be aware that hundreds of comparisons were made in this 
study, and it is probable that some differences between groups were large and statistically 
significant due to chance factors alone. Caution should be used to interpret differences between 
groups as suggestive of a relationship between gain and the factor under consideration rather than 
as conclusive evidence of an effect. 



emphasize statistical significance in this report, and we do not report the 
significance of each comparison. In discussing the results of the survey, we focus 
on sizable differences that also appear to have some practical significance. The 
reader may want to use 20 percent as a rough guide for judging statistical 
significance when making other comparisons of teachers in high-gain and low- 
gain schools. 7 

As is the case with all survey research, several factors may threaten the 
validity of the conclusions reached in this study. There may be selection effects 
because not all principals provided us with teachers' names, and not all teachers 
chose to participate. These refusals may have introduced some degree of bias into 
the reported results. One must also be cautious about self-reported data. 
Respondents may have answered in ways they considered socially desirable, 
leading to results that do not reflect teachers' true beliefs. 

To avoid overly complex language we will often omit explicit reference to 
the self-reported nature of the results. "One-half of the teachers have Masters' 
degrees" is far easier to read than "one-half of the fifth-grade teachers in our 
sample reported that they have Masters' degrees." The reader should remember 
that all these results are based on teachers' survey responses. In addition, we 
occasionally refer to the results for seventh-grade teachers for eighth-grade 
teachers instead of "seventh-grade teachers who teach writing" or "eighth-grade 
teachers who teach mathematics," respectively. The reader should keep in mind 
that we surveyed only writing teachers in seventh grade and only mathematics 
teachers in eighth grade. 

Generalizability of Findings 

The teacher samples are representative of Kentucky fourth-grade teachers, 
fifth-grade teachers, seventh-grade writing teachers and eighth-grade 
mathematics teachers. To test the validity of the sampling process, we compared 
the schools in our samples with those in the state as a whole on three variables: 

7 After the survey was completed, scores for the 1996-97 school year were released, and each 
school's "mid-point" gain score was computed. These gain scores reflect changes after the first year 
of the third two-year cycle, i.e., changes from the baseline established in the 1994-96 biennium. 
This document does not report comparisons based on the mid-point gain scores for a number of 
reasons. Most importantly, Kentucky uses biennium gains to determine rewards and sanctions 
because these scores are more reliable than midpoint gains, and we felt that we should be consistent 
with the state. In addition, we over-sampled based on biennium gains, not midpoint gains, thus we 
have less power to detect differences using midpoint gains, if they existed. 



school size, the baseline accountability index, and the change in the KIRIS 
accountability index during the second biennium. The school-level means were 
similar to the population means for all four samples on these three variables. 
Table 3 contains the sample statistics and the population values for each grade 
level. 



Table 3 

Sample and Population Comparison (Mean Values) 





School 

Size 


1996 Subject Area 
Accountability 
Index 


Gain on Subject 
Area Index 


Sample 


Pop. 


Sample 


Pop. 


Sample 


Pop. 


Fourth-Grade Writing 


71 


69 


47.8 


47.6 


12.5 


12.8 


Fifth-Grade Math 


63 


69 


37.9 


38.8 


12.0 


12.3 


Seventh-Grade Writing 


164 


162 


38.1 


37.2 


1.5 


1.7 


Eighth-Grade Math 


161 


162 


38.4 


39.9 


12.4 


12.8 



Organization of the Report 

The results are organized by subject: mathematics followed by writing. 
Within each subject, the analyses focus on teacher education and experience, 
school and classroom organization, professional development, curriculum 
emphasis, instructional practices, influences on practice and support for change, 
assessment and grading, and portfolios. Information about differences between 
teachers in elementary and middle schools and differences between teachers and 
high-gain and low-gain schools are interwoven in these discussions. The final 
chapter summarizes the results, highlighting comparisons between mathematics 
and writing teachers, elementary school and middle school teachers, and teachers 
in high-gain and low-gain schools. It concludes with a discussion of the 
implications of these results for Kentucky educators and for standards-based 
assessment more broadly. 



MATHEMATICS TEACHING AND LEARNING 



Teacher Education and Experience 

In general, Kentucky fifth-grade teachers and eighth-grade mathematics 
teachers have strong educational backgrounds (albeit in different subject areas), 
considerable teaching experience, and moderate lengths of service in their 
current schools. The majority of the fifth- and eighth-grade teachers surveyed 
reported that they have a master's degree or higher. In fifth grade, 61 percent of 
the teachers have a Masters' degree and 26 percent have a Kentucky Rank One 
Certificate, which requires 30 hours of coursework beyond a Masters' degree. In 
eighth grade, 42 percent of the mathematics teachers have a Masters' degree and 
20 percent have a Kentucky Rank One Certificate. 

The greater academic preparation of elementary school teachers may be due, 
in part, to their longer tenures. The median number of years of teaching 
experience for fifth-grade teachers was 18 years. For eighth-grade mathematics 
teachers, the median length of teaching experience was six years. Fifth-grade 
teachers also had more experience at their current school than eighth-grade 
teachers. The median number of years of teaching experience in their current 
school was 12 years for fifth-grade teachers and five years for eighth-grade 
mathematics teachers. 

As might be expected, the vast majority of the fifth-grade teachers (85 
percent) were certified in Elementary education. However, only 9 percent of fifth- 
grade teachers specialized in mathematics either in their credential preparation 
program or when they renewed their teaching credentials. In contrast, eighth- 
grade mathematics teachers were more diverse in terms of certification and were 
far more likely to specialize in mathematics. Their teaching certificates were 
divided roughly equally among Elementary education (39 percent). Middle 
School (43 percent), and Secondary Grades (30 percent). 8 The majority of the 
eighth-grade mathematics teachers (61 percent) specialized in mathematics in 
their teacher preparation program, and a substantial percentage (45 percent) 
specialized in mathematics when they had their credentials renewed. Overall, 
about three-quarters (72 percent) of the eighth-grade mathematics teachers 
specialized in mathematics during preservice training or credential renewal. 



Teachers can hold teaching certificates in more than one category. 



There was no difference in teaching experience or subject matter 
preparation between teachers in high-gain and low-gain schools in either grade 
level. 



School and Classroom Organization 

Classroom organization for fifth-grade instruction in Kentucky was more 
varied than we anticipated. Only about one-third of Kentucky fifth-grade teachers 
worked in traditional self-contained classrooms. The majority of fifth-grade 
teachers (59 percent) described their school's instructional organization as "semi- 
departmentalized," i.e., students were taught their academic subjects by two or 
three different teachers. This would be the case, for example, if Smith and Jones 
exchanged students for reading and mathematics, with Smith teaching 
mathematics to both groups and Jones teaching reading to both groups. This class 
exchange model applies to at most one-half of the teachers in semi- 
departmentalized schools. The other half of the teachers in semi- 
departmentalized schools reported that they teach all subjects. This might be the 
case if Smith and Jones exchanged some of their students, e.g., a group of 
students of similar ability, with each one teaching mathematics and reading to a 
realigned group or if Smith and Jones team teach some subject area. 

By the time students reach the eighth grade, they are typically assigned to a 
different teacher for each subject. In 83 percent of schools, eighth-grade 
instruction was completely departmentalized, while 17 percent of schools had a 
semi-departmentalized eighth grade. We found no relationships between school 
and classroom organization and KIRIS gains in fifth or eighth grade. 

The majority of teachers at both grade levels reported that their schools did 
not group students by ability when assigning them to classrooms. However, 
ability grouping was almost three times more common at the eighth-grade level 
than at the fifth-grade level. Almost 28 percent of eighth-grade teachers work in 
schools where students were assigned to classes by ability, compared to only 10 
percent of fifth-grade teachers. Ability grouping was equally common in middle 
schools in which the typical eighth-grade student was taking pre-algebra or 
algebra and in middle schools where the typical eighth-grade student was taking 
general mathematics. Ability grouping was not associated with differences in 
KIRIS gain scores at either grade level. 



Ability grouping may also occur within classes, but for the majority of 
teachers at both grade levels it was not a frequent occurrence. Although a slight 
majority of teachers group students by ability in their classes at least occasionally, 
only about one-fourth of the teachers at either grade level indicated that they 
group students by ability "often" or "always." Interestingly, fifth-grade teachers in 
low-gain schools were more likely to use ability grouping than their peers in 
high-gain schools. At the fifth-grade level, 38 percent of teachers in low-gain 
schools reported that they "often" or "always" grouped students by ability, while 
only 12 percent of teachers in high-gain schools did so. This difference did not 
appear in eighth grade. 

Individualized instruction and small group instruction occurs with similar 
frequency in both grade levels. On average, teachers reported that they taught 
mathematics to the whole class 43 percent of the time; they divided students into 
small groups (2-5 students) for mathematics about 36 percent of the time; and 
they worked with students individually about one-quarter of the time. This 
pattern was similar for teachers in low-gain and high-gain schools. 

Professional Development and Preparedness 

Teachers at both grade levels participated regularly in professional 
development in the areas of mathematics instruction and assessment. Teachers 
reported that they benefited from the training they received, and they expressed 
confidence in their ability to teach mathematics. There was little or no difference 
in professional development or preparedness between teachers in high-gain and 
low-gain schools at either grade level. 

Almost all of the teachers surveyed had participated in formal professional 
development activities in the current school year (97 percent) as well as in each 
of the past two years (97 percent and 96 percent for 1995-96 and 1994-95, 
respectively). In addition, during the last three years, 24 percent of the fifth-grade 
teachers and 39 percent of the eighth-grade mathematics teachers acted as trainers 
for professional development in mathematics. 

Teachers participated in professional development that focused on 
instructional goals and outcomes, mathematics curriculum, and the KIRIS 
assessments. Overall, two-thirds or more of the teachers participated in 
professional development that focused "a moderate amount" or "a great deal" 
on the following topics: mathematics curriculum and instruction (77 percent). 



KERA goals and outcomes (76 percent), preparing students for the KIRIS 
assessments (71 percent), and preparing students for mathematics portfolios (68 
percent). In contrast, fewer than one-half participated in professional 
development that focused strongly on the following topics: classroom 
assessment methods (41 percent), curriculum and instruction in other subjects 
(40 percent), use of educational technology (38 percent), and Language Arts 
curriculum and instruction (32 percent). 9 

Teachers in both grades indicated that the professional development they 
received helped them with their mathematics teaching. Specifically, the majority 
of teachers reported that professional development helped them either "a 
moderate amount" or "a great deal" to improve their ability to: help students 
with their mathematics portfolios (71 percent and 58 percent in fifth and eighth 
grade, respectively), prepare for KIRIS open-response items in mathematics (57 
percent and 69 percent in fifth and eighth grade, respectively), and use 
manipulatives to teach mathematics (61 percent and 52 percent in fifth and 
eighth grade, respectively). In addition, 51 percent of the fifth-grade teachers 
indicated that professional development helped them to teach mathematical 
communication (compared to 36 percent of the eighth-grade mathematics 
teachers). 

Fifth-grade teachers and eighth-grade mathematics teachers expressed high 
levels of confidence in their preparedness to teach a variety of mathematics 
topics. Table 4 shows the percent of teachers in each grade level who felt "fairly 
well prepared" or "very well prepared" to teach the topics in the Kentucky 
mathematics curriculum. The vast majority of teachers in both grades felt 
prepared to teach most of the topics, but only a small majority felt prepared to 
teach technology. Fewer fifth-grade teachers than eighth-grade teachers felt 
prepared to teach statistics and probability and algebraic ideas, although this is 
part of the elementary curriculum in Kentucky. 



9 As one might anticipate, there were differences between fifth-grade teachers and eighth-grade 
mathematics teachers in participation in professional development on Language Arts curriculum 
and instruction. Forty-nine percent of fifth-grade teachers participated in this type of training 
compared to 15 percent of eighth-grade mathematics teachers. 



Table 4 



Teacher Preparation for Mathematics Topics (Percent of Teachers Who Were "Fairly Well" or 
"Very Well" Prepared) 



Topic 


Fifth 

Grade 


Eighth 

Grade 


Numbers and computation 


94 


96 


Geometry and measurement 


86 


94 


Statistics and probability 


52 


86 


Algebraic ideas 


54 


93 


Connections among mathematical ideas 


82 


94 


Connections between mathematics and other subjects 


81 


84 


Problem solving 


86 


96 


Mathematical communication 


81 


92 


Reasoning 


79 


89 


Technology (calculators and computers) in support of mathematics 


52 


62 



There were mild associations between KIRIS gains and teacher 
preparedness. In fifth grade, a larger percentage of teachers from high-gain 
schools than low-gain schools expressed confidence in their preparation to teach 
algebraic ideas (67 percent compared to 49 percent). Similarly, at eighth grade, a 
larger percentage teachers in high-gain schools than in low-gain schools felt 
prepared to teach statistics and probability (94 percent compared to 73 percent). 

Curriculum Emphasis 

We asked somewhat different questions about curriculum emphasis to 
fifth- and eighth-grade teachers. Those fifth-grade teachers who taught all 
subjects were asked about their allocation of instructional time across subject 
areas and the degree to which they integrated mathematics with other subjects. 
Eighth grade teachers were asked about the degree to which they work with 
teachers in other subject areas. Both fifth- and eighth-grade teachers were asked 
about their allocation of instructional time to various aspects of mathematics. 



Allocation of Instructional Time Across Subjects 10 

Originally, the KIRIS assessments in all subjects were given in fourth grade 
(in elementary schools). However, this created too great a burden on fourth- 
grade students and teachers, so the assessments were divided between grades 
four and five. Fourth-grade students are tested in reading, writing and science. 
Fifth-grade students are tested in mathematics, social studies, arts and 
humanities, and practical living/ vocational studies. This change occurred in 
1996-97. We were interested in the impact of the KIRIS assessments on the 
allocation of instructional time, and whether there was an association between 
the subjects that were assessed and the subjects emphasized by teachers. 

Fifth-grade teachers, who teach all subjects, reported devoting as much time 
to mathematics as to reading. Writing, social studies and science received 
somewhat less instructional time, and teachers devoted the least time to Arts & 
Humanities and Practical Living/Vocation Studies (see Table 5). In addition, they 
reported that instructional time had stayed the same or increased in almost all 
cases. However, teachers were more likely to report increases in subjects 
associated with KIRIS than in other subjects. 

Two-thirds of the fifth-grade teachers reported increasing the amount of 
instructional time they spend on mathematics (and none of the teachers reported 
a decrease in time spent on mathematics). More than one-half of the teachers 
also increased the amount of time they devoted to writing. Arts and Humanities, 
and Practical Living/Vocational Studies. Although the writing assessment 
remained in grade 4, it is not surprising that fifth-grade teachers increased 
writing instruction as the KIRIS assessments in all subject areas require a great 
deal of writing, particularly the mathematics portfolios. Social studies was the 
only subject area tested in fifth grade for which a majority of the teachers did not 
report an increase. There were only two subjects for which a substantial number 
of fifth-grade teachers reported a decrease in instructional time. Eighteen percent 
of fifth-grade teachers said they reduced the time devoted to reading and 10 
percent reduced the time devoted to science. -- < 



10 Thirty-two percent of fifth-grade teachers indicated that they did not teach all subjects and 
skipped the questions about the allocation of instructional time across subjects. Thus, they were 
excluded from the results reported in this section. 



Table 5 

Fifth-Grade Subject Coverage and Change in Coverage (Percent of Teachers 
Reporting Increase in Coverage During Past Three Years) 



Subject 


Median 
Hours per 
Week 


Percent 

Increasing 

Coverage 


Mathematics 4 


5 


66 


Writing 


4 


61 


Arts and Humanities 4 


2 


58 


Practical Living/Vocational Studies 4 


1 


52 


Science 


4 


39 


Social Studies 4 


4 


36 


Reading 


5 


10 



* Denotes subject tested in KIRIS in grade five; other subjects are tested in grade 
four. 



Some caution is warranted in interpreting the results regarding total class 
time per subject, although there is little reason to question the relative changes 
in time among subjects. Seventy-nine percent of teachers reported an increase in 
class time for at least one subject, but only 26 percent reported a decrease in class 
time for any subject. In fact, many teachers reported increasing the time they 
devoted to all subjects. The case studies of exemplary teachers conducted as part 
of this research provide two possible explanations for this increase. One 
possibility is that by integrating subjects teachers increase the class time they 
devote to each; the other possibility is that teachers have increased students' 
overall time on task by reducing non-academic classroom activities. 

We collected additional information about the integration of mathematics 
with other subjects. Fifth-grade teachers reported that they often integrated 
mathematics with other subjects and that the frequency of integration has 
increased. The vast majority of teachers reported integrating mathematics with 
each of the following subjects at least once a month: reading (85 percent); writing 
(95 percent); social studies (77 percent); and science (92 percent). In fact, the 
majority of teachers (67 percent) reported integrating mathematics with writing 
at least once a week, which may be due to the mathematics portfolios. 



For each subject area we asked about, except arts and humanities, at least 
one-half of the teachers reported that they have increased the frequency with 
which they integrate the subject with mathematics (reading 58 percent, writing 73 
percent, social studies 50 percent, science 53 percent, and practical 
living /vocational studies 50 percent). There were no consistent differences 
between fifth-grade teachers in high-gain and low-gain schools in the frequency 
of integrating mathematics with other subjects. 

Allocation of Instructional Time within Mathematics 

There has been considerable change in emphasis within mathematics at 
both grade levels. Overall, 80 percent of the teachers reported that the content of 
their mathematics teaching had changed either a moderate or a major amount 
during the past three years; not a single teacher reported no change. Numbers 
and computation receives the greatest share of class time in both fifth and eighth 
grades with 95 percent of teachers reporting that they cover the content area at 
least once a week (see Table 6). In fifth grade, no other content area is covered as 
frequently. However, in eighth grade, 80 percent of the teachers reported 
covering algebraic ideas at least once a week. 



Table 6 



Frequency of Mathematics Content Area Coverage (Percent of Teachers Who 
Covered Content at Least Once a Week) 



Content area 


Fifth Grade 


Eighth Grade 


Number and computation 


95 


95 


Geometry and measurement 


39 


39 


Algebraic ideas 


30 


80 


Statistics and probability 


13 


24 



Table 7 shows the change in the frequency with which content areas are 
covered. More than one-half of the teachers in both .grades increased , the time 
devoted to statistics and probability and to algebraic ideas. 



Table 7 

Change in Mathematics Content Area Coverage (Percent of Teachers Who 
Increased Coverage During the Past Three Years) 



Content area 


Fifth Grade 


Eighth Grade 


Number and computation 


10 


20 


Geometry and measurement 


46 


44 


Statistics and probability 


61 


54 


Algebraic ideas 


59 


54 



At both grade levels teachers from high-gain schools reported a higher 
frequency of coverage in "algebraic ideas" than did teachers from low-gain 
schools (45 percent vs. 25. percent for fifth-grade and 88 percent vs. 60 percent for 
eighth-grade teachers). In eighth grade, a greater percentage of teachers in high- 
gain schools than low-gain schools reported increasing their coverage of all four 
mathematics topics. For example, 75 percent of teachers from high-gain schools 
increased coverage of "statistics and probability," compared to only 40 percent of 
teachers from low-gain schools. Surprisingly, in fifth grade more teachers from 
low-gain schools than high-gain schools increased their frequency of teaching 
"statistics and probability" (69 percent and 46 percent, respectively). However, 
since few fifth-grade teachers covered this content on a regular basis, these 
differences should not be over-interpreted. 

Instructional Practices 

Teachers identified themselves with standards-based approaches to teaching 
mathematics, and they used both traditional, and standards-based strategies for 
teaching. They also assigned both types of activities for students. Traditional 
approaches were somewhat more common, and used more frequently. However, 
during the past three years a large majority of teachers increased their use of 
standards-based approaches. There were few differences between elementary and 
middle schools in this - regard, -and there -was— little - relationship between 
instructional practices and KIRIS gains in the second biennium. 

Philosophy and Approach to Teaching Mathematics 

Teachers characterized their overall approaches to teaching mathematics on 
four dimensions defined by contrasting statements: one end of each dimension 



was more consistent with a standards-based approach; the other with a 
traditional approach. The four dimensions concerned goals for students, 
philosophy of instruction, the content of mathematics, and the order of 
presentation. Teachers indicated whether they emphasized thinking and 
questioning mathematically vs. mastering the content of mathematics, learning 
terms and computation vs. conceptual understanding, in-depth study of a few 
topics vs. comprehensive coverage of many topics, and teaching facts before 
principles vs. teaching principles before or simultaneously with facts. Each 
dimension had five levels, including one that was midway between the two 
descriptors. Teachers marked one of five points that reflected their approach. 

On three of the four dimensions, between one-half and two-thirds of the 
teachers rated themselves closer to the standards-based end of the continuum. 
For example, 77 percent of fifth-grade teachers and 68 percent of eighth-grade 
mathematics teachers placed their approach closer to the statement that a teacher 
should primarily "encourage students to think and question mathematically" 
than to the statement that a teacher should primarily "transmit the knowledge 
and content of mathematics." One-half or more of the teachers at both grade 
levels also identified more closely with conceptual understanding than with 
learning mathematical terms and computation, and with teaching principles 
before (or with) facts than with teaching facts before principles. 11 On the fourth 
dimension — depth vs. breadth — the largest number of teachers rated themselves 
midway between the two extremes. There were no significant differences in 
overall teaching approach between teachers in high-gain and low-gain schools. 

Teachers' responses were similar when they were asked more specific 
questions about their teaching methods. They indicated support for both 
traditional and standards-based activities. For example, more than 98 percent 
agreed with the traditional idea that students require a good deal of practice to 
become competent in mathematics. Similarly, more than 75 percent agreed that 
students must learn facts, principles, and formulas, and that they should always 



11 Among fifth-grade teachers, 54 percent placed themselves closer to conceptual understanding 
while five percent placed themselves closer to mathematical terms and computation. Forty-two 
percent placed themselves midway between the two ends. Fifty eight percent aligned themselves 
more with teaching principles before (or with) facts, compared to 15 percent who supported 
teaching facts before principles. Among eighth-grade teachers, 53 percent were closer to conceptual 
understanding compared to eight percent who were closer to mathematical terms and computation; 
53 percent were more aligned with teaching principles before (or with) facts compared to 19 percent 
who placed themselves closer to teaching facts before principles. 



show their work. However, all of the teachers also agreed with the standards- 
based viewpoint that different students learn to solve problems in different ways, 
65 percent agreed that good teachers create their own lessons and materials, and 
more than three-quarters disagreed that "teachers should follow the 
mathematics textbook." 

These results show that the standards-based and traditional approaches are 
not mutually exclusive in teachers' minds. They endorse practices associated 
with each approach. However, teachers also revealed some uncertainty about 
how to integrate the two. approaches in particular circumstances. For example, 
Table 8 shows teachers' responses to four questions about the degree to which 



Table 8 

Teachers' Opinions About Student Learning in Mathematics (Percent of Teachers Who Agreed 
with Statement) 





Fifth Grade 


Eighth Grade 


Statement 


Some- 

what 

Agree 


Strongly 

Agree 


Some- 

what 

Agree 


Strongly 

Agree 


Students' errors should be corrected quickly so they 
do not finish a lesson feeling confused or stuck 


27 


62 


43 


42 


Students should receive step by step directions to 
aid in problem solving 


49 


28 


44 


27 


Teachers should not necessarily answer students' 
questions, but should let them figure things out 
themselves 


75 


16 


65 


17 


Students learn best if they have to figure things out 
for themselves instead of being told or shown what 
to do 


53 


26 


51 


20 



students should be left to puzzle things out for themselves. The large majority of 
teachers agreed with the four statements, although the statements carry 
somewhat" contradictory' messages. On' the one “hand, "teachers endorsed the 
notion that students should be allowed to figure things out for themselves. On 
the other hand, they believed students should be given step-by-step instructions 
and should have their errors corrected quickly, which would preclude some 
students from solving problems on their own. There were strong similarities on 



these questions between the responses of fifth- and eighth-grade teachers, and 
between teachers in high-gain and low-gain schools. 

Expectations for Students 

A key assumption of the KERA reform is that all students can learn to high 
levels. While teachers endorse higher expectations for students, in general, they 
do not believe that all students can learn to the same high levels. Moreover, they 
agree that too much pressure can squelch students' enthusiasm for mathematics. 
Table 9 shows that two-thirds of the teachers believe students can learn more 
than they have been expected to learn in the past. However, only about one-half 
believe all students can learn mathematics at grade level, and about two-thirds of 
the teachers agreed that there are some students who will never perform above 
the level of Novice. 12 



Table 9 

Teachers' Expectations for Students (Percent of Teachers Who Agreed "Somewhat" or "Strongly" 
with Statement) 





Fifth 


Eighth 


Statement 


Grade 


Grade 


All students can learn more than we have been teaching them in the past 


67 


73 


There are some students who will never perform above the level of 
Novice, no matter how much effort the teacher devotes to them 


63 


70 


I believe all students can learn fifth [eighth] grade mathematics by the 
end of fifth [eighth] grade 


48 


48 


Teachers should use the same standards in evaluating the work of all 
students in class 


35 


45 



It is likely that the kinds of interventions that help students move from 
Novice work to Apprentice work are different than the kinds of interventions 
that help them move from Apprentice to Proficient or from Proficient to 
Distinguished. The majority of fifth-grade teachers (59 percent) reported that 
moving students from- Proficient to- Distinguished is-more difficult than moving 
students up from other levels. Forty percent of eighth-grade teachers agreed that 
moving students from Proficient to Distinguished is most difficult. However, 



12 Student performance is assigned to one of four categories: Novice, Apprentice, Proficient, and 
Distinguished. 



one-quarter of the eighth-grade teachers reported that all of the steps are equally 
difficult, while only 10 percent of fifth-grade teachers responded this way. 

Teachers appear to be trying to find a balance between high expectations and 
putting too much pressure on students. Eighty-two percent agreed that teachers 
in their school push students fairly hard in academic subjects. The majority (70 
percent) also agreed that too much pressure can cause students to lose their 
enthusiasm for mathematics. Overall, there were no differences in expectations 
between the grade levels or between teachers in high-gain or low-gain schools. 

Teaching Strategies 

Teachers reported regular use of both traditional and standards-based 
teaching strategies in mathematics, and they generally reported an increase in the 
frequency of standards-based practices (see Table 10). More than 80 percent of the 
fifth-grade teachers used four of the eight strategies at least weekly, and more 

Table 10 

Use of Teaching Strategies (Percent of Teachers Who Used Strategy at Least 1-2 Times per Week and Percent Who 
Increased Use During Past Three Years) 

Fifth Grade Eighth Grade 



Strategy 


Weekly 

Use 


3-year 

Increase 


Weekly 

Use 


3-year 

Increase 


Traditional approaches 










Demonstrate how to perform a new mathematics skill 


96 


18 


91 


17 


Explain correct solutions to assigned problems 


89 


16 


100 


9 


Explain a new concept 


80 


18 


78 


14 


Give tests or quizzes 


42 


4 


46 


6 


Standards-based approaches 










Ask open-response questions with many right answers 


61 


88 


44 


70 


Give examples of real-world applications of mathematics skills 


91 


46 


83 


46 


Demonstrate mathematical ideas using objects, constructions, etc. 


55 


53 


61 


41 


Show connections between mathematics and other subjects 


68 


33 


55 


39 



than one-half used seven of the eight at least weekly. The pattern was similar for 
eighth-grade teachers, but there were some differences. Fifth-grade teachers were 
more likely to ask open-ended questions and show connections between 



mathematics and other subjects than eighth-grade teachers (61 vs. 44 percent, and 
69 vs. 55 percent, respectively). 

At both grade levels, the greatest increases in use occurred for the four 
standards-based strategies. For example, the vast majority of teachers increased 
their emphasis on open-response questions with many right answers. About 
one-half of the teachers also increased their use of real-world applications, 
demonstrations using objects, and connections with other subjects. So, although 
traditional approaches continued to be used on a regular basis, the greatest 
change in practice was in the direction of standards-based approaches. 

The largest difference in teaching strategies between teachers in low-gain 
and high-gain schools was in the area of testing. A greater percentage of eighth- 
grade mathematics teachers from high-gain schools (80 percent) than low-gain 
schools (35 percent) gave tests or quizzes at least once a week. The difference was 
smaller, among fifth-grade teachers. Forty-one percent of teachers in high-gain 
schools gave tests at least weekly compared to thirty percent of teachers in low- 
gain schools. 

In addition to questions about general mathematics teaching strategies, we 
asked teachers about three specific instructional scenarios: how teachers allocate 
time across subtopics within the general theme of graphing, how teachers would 
respond to errors made by students while working at the board on computations 
with fractions, and how teachers would begin and end an instructional unit on 
summarizing data. These three topics were selected because they are part of the 
normal curriculum at both grade levels and are consistent with the learning 
outcomes delineated in the Kentucky transformations and the core concepts for 
mathematics. Teachers' reports about specific mathematics instructional 
activities revealed considerable uniformity of practices across grade levels and 
between high-gain and low-gain schools within a grade level. 

Both fifth- and eighth-grade teachers reported that they gave graphing 
considerable attention. 13 In both grades, teachers spent 20 class hours (on average) 
and assigned about 12 homework ^assignments *(ori ’ average) dealing with 
graphing (and data displays). Most teachers in both grades reported that a 
moderate or a great deal of time was spent on "traditional" graphing activities, 



13 Considerable attention was paid to "graphing" in fifth grade and "graphing and data displays" 
in eighth grade. 



including producing graphs (of different types) and collecting data and displaying 
it in graphs (see Table 11). In fifth grade, most teachers also devoted a moderate 
amount or a great deal of time to three of the four activities that are consistent 
with the standards-based approach of KERA. In eighth grade, the majority of 
teachers devoted a moderate or a great deal of time to two of the four standards- 
based activities. 



Table 11 

Classroom Activities Involving Graphing (and Data Display) (Percent of Teachers Who Devoted 
a "Moderate Amount" or "A Great Deal" of Time to Activity) 



Activity [grade level] 


Fifth 

Grade 


Eighth 

Grade 


Traditional approaches 






Producing bar graphs and pie (circle) graphs [5]; producing tables and 
graphs (line graphs, box and whisker plot s, etc.) [8] 


74 


82 


Collecting data and displaying it in graphs [5]; collecting data, selecting 
an appropriate format and producing a table or graph [8] 


75 


76 


Standards-based approaches 






Interpreting graphs from different sources (e.g., newspapers and 
magazines) 


70 


41 


Producing graphs for assignments in other subjects (e.g., social studies) 


62 


39 


Solving word problems in which important information is contained in 
graphs 


78 


57 


Constructing graphs or data displays using objects, construction papers, etc. 


41 


50 



In both grades teachers in low-gain schools reported spending somewhat 
more time on graphing (and data displays) than teachers in high-gain schools. 
The typical (median) teacher in a low-gain school spent about 20 class periods on 
graphing while the typical teacher in a high-gain school spent about 15. In eighth 
grade the emphasis on graphing was related to the mathematics course taken by 
the typical student. In schools where the average eighth-grade student enrolled 
in pre-algebra or algebra- (based'on teachers i reports), J the typical teacher spent 13 
periods on graphing. In schools where the average eighth-grade student enrolled 
in eighth-grade mathematics, the typical teacher spent 20 periods on graphing. 

The relative emphasis on different aspects of graphing was similar for 
teachers in high- and low-gain schools. This was true for all the activities 



27 



31 



identified as "traditional" and most of those identified as "non-traditional." The 
only exceptions were "constructing graphs or data displays using objects" in fifth 
grade (39 percent of teachers in low-gain schools spent a moderate amount of 
time or a great deal of time on this compared to 56 percent in high-gain schools) 
and "solving word problems in which important information is contained in 
graphs" in eighth grade (45 percent of teachers in low-gain schools devoted a 
moderate or great deal of time to this compared to 77 percent of teachers in high- 
gain schools). 

The second instructional activity we addressed was responding to student 
errors; specifically we asked teachers how they would react in a hypothetical 
situation in which a student made a mistake while working at the board. For 
example, fifth-grade teachers responded to the following situation: 

Imagine you have been teaching a unit cn reducing fractions to lowest terms for about a 
week. You ask for volunteers to come to the board to write 18/24 in lowest terms. A 
student of average ability comes to the board and writes 18/24 = 9/12. How likely are 
you to respond in the following ways? 14 

The pattern of responses to these situations was almost identical across the 
grade levels, with teachers preferring, in order, actions that drew information 
from the students at the board, from other students in class, and, least frequently, 
providing information themselves. 15 Table 12 shows the percentage of teachers 
in each grade indicating they were "somewhat likely" or "very likely" to take 
each action. Almost all teachers said they would be most likely to respond in 
ways that drew information from students. For example, most teachers reported 
it was likely that they would ask the student another question to lead to the 
answer or that they would call on other students to help out. This is consistent 
with the "student-centered" approach embodied in KERA. Although they were 
less frequent, responses in which the teacher supplied the missing information 
were also common. For the most part, there were only minor differences 
between grade levels, and, within grade levels, there were only minor differences 
between teachers in-low-gain--and -high-gain schools. -- - 



14 Eighth-grade teachers responded to a similar hypothetical situation in which the problem 
involved dividing by a fraction. On the eighth-grade survey, the student's solution to the problem 
1/2 + 1/4 is correct but the student remarks, "That's wrong. The answer is larger than the numbers I 
started with, but division is supposed to give smaller answers." 

15 The response options were similar for fifth- and eighth-grade teachers with only minor changes 
reflecting the differences in problems. 



Table 12 

Responding to Student Errors (Percent of Teachers Who Were "Somewhat Likely" or "Very 
Likely" to Respond in a Given Manner) 



Teacher response [grade level] 


Fifth 
: Grade 


Eighth 

Grade 


Focus on student at the board 






Ask student to explain his/her reasoning 


95 


95 


Ask student another question to lead to the answer 


92 


99 


Suggest a concrete method (suggest drawing a picture of fractions [5]; 
suggest using blocks or objects to solve [8]) 


41 


60 


Involve other students 






Ask all the students if they agree with answer 


81 


86 


Ask other students how one could check the answer 


76 


76 


Obtain different responses from students and discuss which is correct 


70 


70 


Ask other students about reasons for error 


— 


. 72 


Have all students work problem on own 


52 


— 


Thank the student and call on another for a different solution 


24 


6 


Provide information themselves 






Point out the error (indicate 9/12 is not in lowest terms [5]; explain that 
when dividing by 1 /4 the quotient should be larger [8]) 


63 


42 


Review relationship between multiplication and division 


— 


71 



The third aspect of instruction teachers described was strategies for 
beginning and ending a new curriculum unit. Teachers indicated how they 
would initiate a unit on the topic of summarizing data (averages in fifth grade, 
measures of central tendency in eighth grade) and what sort of culminating 
activity they would use. Options included "traditional” activities, such as 
teaching the procedures for computing averages and giving a test, as well as 
"reform-oriented" activities, such as posing a problem and assigning a portfolio 
prompt. 

Fifth-grade teachers were more likely to begmlhe unit In "non-traditional" 
ways than traditional ways. For example, 80 percent of fifth-grade teachers were 
"somewhat likely" or "very likely" to begin the unit by discussing the concept of 
average as it applies in another subject. Similarly, about 70 percent were likely to 
give a problem whose solution involves an average or ask students to estimate a 



typical quantity (see Table 13). The computational approach ("explain how to 
compute an average") was the fourth most likely strategy for initiating the unit. 
In comparison, eighth-grade mathematics teachers were about equally likely to 
adopt the traditional approach ("Explain the difference between mean, median, 
and mode and how to calculate each") as they were to adopt a non-traditional 
approach ("have students graph features of their classmates" or "have students 
graph features of their classmates and discuss the differences between the 
graphs"). 



Table 13 

Beginning a Curriculum Unit (Percent of Teachers Who Were "Somewhat Likely" or "Very 
Likely" to Respond in a Given Manner) 



Teacher response [grade level] 


Fifth 

Grade 


Eighth 

Grade 


Traditional approaches 






Explain how to compute an average [5]; explain the difference between 
mean, median and mode and how to calculate each [8] 


55 


78 


Standards-based approaches 






Discuss the concept of average as it applies in another field [5]; discuss 
measures of central tendency as they apply in another subject [8] 


80 


69 


Pose a problem that can be solved using averages [5]; pose a related, open- 
ended problem that involves measures of central tendency [8] 


72 


74 


Have students estimate the shoe size of a typical student (or some 
similar quantity) [5]; have students graph features of their classmates 
and discuss the differences between the graphs [8] 


68 


83 


Give students four towers of blocks and ask them to re-arrange the blocks 
to even out the towers 


51 


— 


Show students a graph of the height of various buildings and ask what is 
typical [5]; show students a table of information about enrollments in 
Kentucky middle schools and ask what is typical [8] 


48 


54 


Have students draw bar and line graphs and identify mean, median and 
mode 


— 


69 


Ask students to write a story describing the "typical" student 


— 


28 



The difference between fifth-grade teachers and eighth-grade mathematics 
teachers is reinforced if we concentrate only on the strategies teachers are "very 
likely" to use. Over one-half of the eighth-grade teachers are very likely to begin 
the unit using the traditional approach (54 percent) compared to one-third of the 



fifth-grade teachers (33 percent). Within grade levels there were only minor 
differences between teachers in low-gain and high-gain schools. 

At the end of the hypothetical unit on averages most teachers said they 
would have a culminating assessment activity, and most of these were consistent 
with the assessment components of KIRIS. Fifth-grade teachers were likely to 
assign a set of open-response questions (92 percent) or assign a portfolio prompt 
(87 percent). Many fifth-grade teachers were also likely to assign a set of exercises 
(71 percent) or to give a test (67 percent). Only 10 percent said they were likely to 
end the unit without a special culminating activity. Eighth-grade mathematics 
teachers were more likely to end the unit by assigning a portfolio prompt (92 
percent) or giving a test (86 percent). 16 About one-half of the eighth-grade 
mathematics teachers said they were likely to end the unit with a set of exercises; 
fewer than five percent said they would have no special culminating activity. 

There was one substantial difference between eighth-grade mathematics 
teachers in high-gain and low-gain schools concerning culminating activities for 
a unit on measures of central tendency. A greater percentage of mathematics 
teachers in high-gain schools (98 percent) were likely to give a test than were 
mathematics teachers in low-gain schools (72 percent). 

Student Learning Activities 

Teachers ask students to do both traditional and standards-based learning 
activities regularly, but they have increased the frequency with which students 
do things consistent with the standards-based approach. The most common 
activities across the grade levels were practicing computation skills, working 
problems from the textbook, explaining why students solved a problem a certain 
way, and using calculators and computers to solve mathematics exercises (see 
Table 14). 

The majority of teachers in both grade levels increased the frequency of 
most standards-based activities while maintaining the frequency of traditional 
activities. For three traditional activities, a substantial percentage of teachers 
reported a decrease in the frequency of the activity: Thirty-five percent of teachers 
decreased the amount of time they spend on working problems from the 
textbook; 30 percent cut back on the amount of time dedicated to memorizing 



16 Eighth-grade mathematics teachers were not asked about assigning open-response questions. 



mathematics facts, rules, or formulas; and another 18 percent reduced time spent 
on practicing computation skills. 



Table 14 

Student Learning Activities (Percent of Teachers Who Assigned Activity at Least 1-2 Times per 
Week and Percent Who Increased Use During Past Three Years) 





Fifth Grade 


Eighth Grade 


Activity 


Weekly 

Use 


3-year 

Increase 


Weekly 

Use 


3-year 

Increase 


Traditional approaches 










Practice computation skills 


97 


6 


86 


u 


Work problems from the textbook 


83 


3 


79 


6 


Memorize mathematics facts, rules or formulas 


33 


7 


33 


9 


Standards-based approaches 










Explain why they solved a problem a certain way 


73 


57 


70 


53 


Use mathematics to solve real-world problems 


73 


56 


64 


39 


Use calculators or computers to solve exercises 


65 


68 


91 


47 


Write about mathematics 


61 


82 


41 


65 


Represent concepts or ideas in tables, graphs or 
pictures 


53 


56 


37 


49 


Solve problems using manipulatives 


54 


63 


40 


56 


Use mathematics in the content of [5], integrate 
mathematics with [8] other subjects 


46 


44 


24 


37 


Use measuring tools in mathematics 


37 


29 


30 


30 


Discover mathematics concepts for themselves 


35 


38 


27 


42 


Solve non-routine problems 


29 


59 


27 


36 


Work on extended mathematics activities that 
take several days 


27 


65 


10 


48 



Table 14 is also notable for the consistency in reported practice between 
grade levels. For the most part, fifth and ejghth-_grade teachers reported engaging 
students in various activities at similar levels. There were a few exceptions, 
some of which may be explained by differences in the content and organization 
of the curriculum in the two grades. For example, the biggest difference was in 
the use of calculators or computers to solve exercises, with eighth-grade teachers 
more likely than fifth-grade teachers to engage in that activity at least weekly (91 



percent compared to 65 percent). The other differences were in the frequency of 
writing about mathematics and the frequency of working on extended 
mathematics activities. Forty-one percent of eighth-grade teachers compared to 
60 percent of fifth-grade teachers had students write about mathematics at least 
weekly. The comparable figures for extended mathematics activities were 10 
percent in eighth grade compared to 27 percent in fifth grade. There were no 
sizable differences in the frequency of student learning activities between 
teachers in high-gain and low-gain schools in either grade level when comparing 
use on a weekly basis. However, in eighth grade, there were differences between 
high-gain and low-gain schools in the percent of teachers who used certain 
learning activities on a daily basis. Mathematics teachers in high-gain schools 
were more likely to have students practice computation skills on daily basis than 
teachers in low-gain schools (67 percent compared to 39 percent), and they were 
more likely to have students use calculators or computers to solve exercises on a 
daily basis, as well (53 percent compared to 32 percent). 

Influences on Practice and Support for Change 

KIRIS 

Teachers were divided in their appraisal of the effects of KIRIS on 
mathematics instruction. About one-half of the teachers (54 percent) said that 
KIRIS had a negative impact compared to about one-third (36 percent) who felt 
KIRIS has had a positive effect. This pattern was similar for fifth- and eighth- 
grade teachers. 

However, there were differences in the responses from eighth-grade 
teachers in high-gain and low-gain schools. Eighth-grade teachers in schools 
where the mathematics scores had increased the most during the second 
biennium were much more positive about the impact of KIRIS than teachers in 
schools where the scores had decreased or increased only a small amount. Forty- 
six percent of the eighth-grade mathematics teachers in high-gain schools felt 
that KIRIS had a positive impart on their school's mathematics program 
compared to only 19 percent of teachers in low-gain schools. This difference did 
not occur among fifth-grade teachers in high-gain and low-gain schools. 

Conversely, most teachers do not believe that their school's mathematics 
program has produced broad improvements in student learning that have a 
large impact on KIRIS scores. Teachers who reported that their schools' KIRIS 



mathematics scores had increased during the second biennium identified the 
factors that contributed "a great deal" to the score gain. Table 15 compares the 
responses of teachers in 1996-97 (after score gains from the second biennium had 
been reported) to the responses of teachers in 1994-95 (Koretz et al., 1997). 17 
Although the factors thought by most teachers to contribute "a great deal" to 
KIRIS gains have changed, what has not changed is the belief that gains are not 
associated strongly with improvements in students' mathematics knowledge and 
skills. In 1996-97, the majority of fifth-grade teachers and eighth-grade 
mathematics teachers reported that differences between cohorts of students had 
"a great deal" of influence on their schools' gain in scores. A lower percentage of 
the teachers reported that testwiseness (including familiarity, test-taking skills, 
and practice tests) contributed a great deal to score gains. Only a small percentage 
of teachers thought that their school's KIRIS mathematics gains could be 
attributed to improved knowledge and skills — either the specific skills 
emphasized by KIRIS (8 percent) or mathematics knowledge and skills more 
broadly (8 percent). Presumably teachers have greater experience with KIRIS in 
1996-97 than they did in 1994-95. Many appear to believe that factors outside their 
control (i.e., differences between students) have a stronger influence on score 
gains than factors within their control. Even within the domain of their 
classroom, teachers think that scores are more a function of test taking skills and 
specific test preparation activities than improvements in students' knowledge of 
mathematics. 

One possible explanation for these results is that students are coming into 
fifth grade and eighth grade with greater knowledge of mathematics, i.e., KERA 
has had a positive effect in the earlier grades. Fifth-grade teachers and eighth- 
grade mathematics teachers might describe this situation in terms of better 
students rather than better preparation. However, if this were the case, one 
would expect them to attribute more of the gains to mathematics knowledge and 
skills, which they do not do. 



17 The results represent the combined responses of teachers in the mathematics accountability 
grades in elementary and middle schools (fifth-grade teachers and eighth-grade mathematics 
teachers in 1996-97; fourth-grade teachers and eighth-grade mathematics teachers in 1994-95). 



Table 15 

Reasons for KIRIS Score Gains in Mathematics (Percent of Teachers Who Reported that Each 
Factor Contributed "A Great Deal" to KIRIS Score Gains in their School) 



Factor 


1996-97 


1994-95 


Improved student test taking skills 


12 


34 


Increased student motivation 


5 


16 


Differences between groups of students from one year to the next 44 


56 


26 


Broad improvements in students' mathematics knowledge and skills 4 


8 


16 


Specific improvements in student mastery of mathematics knowledge and 
skills emphasized by KIRIS 44 


8 


24 


Increased student familiarity with the KIRIS mathematics assessments 4 


18 


55 


Students' work with practice tests and other preparation materials in 
mathematics 4 


10 


51 



* The words "student(s)" and "mathematics" were added to these items in 1996-97. 

* * Other minor wording changes were to these items. 



State and Local Support for Reform 

Over the past few years, the Kentucky Department of Education has 
invested large amounts of resources in support of educational reform in 
Kentucky. KDE has developed academic expectations, created and improved 
KIRIS, developed supportive curriculum materials, sponsored workshops on 
scoring and using KIRIS assessments, and provided extensive opportunities for 
professional development. Other agencies, such as Kentucky Educational 
Television, local school districts, and individual schools, have also contributed 
to the effort to promote reforms consistent with KERA. Kentucky teachers have 
had access to a wealth of relevant information and support. 

Teachers found that some factors had a greater influence than others on the 
content of their mathematics lessons and the methods they used to teach 
mathematics. We asked about seventeen different items that were grouped into 
four broad clusters: curriculum guides and textbooks, assessments, school staff, 
and professional development. We asked separately about the influence of these 
items on what they teach and how they teach. In general, teachers responded 
similarly to these two questions, so we have chosen to present only the results 
for factors that influence what they teach. 



Five of the 17 items had “a great deal of influence" on what was taught by 
more than one-half of the teachers. These included three curriculum documents: 
Transformations (KDE, 1995a), Core Content for Mathematics Assessment (KDE, 
undated), and Curriculum and Evaluation Standards for School Mathematics 
(NCTM, 1989) and two assessment-related items: the KIRIS mathematics 
portfolio, and the KIRIS open-response mathematics items. Almost three- 
quarters of teachers reported that Core Content for Mathematics Assessment had 
a great deal of influence over the content of their teaching compared to about 50 
percent for the other two curriculum documents. Sixty-six percent of teachers 
said the KIRIS open-response mathematics items had a great deal of influence on 
the content of their teaching, as did 57 percent regarding the KIRIS mathematics 
portfolio. 

District curriculum guides, supplemental mathematics materials, and 
"other" professional development were the next most influential, exerting a 
great deal of influence on what was taught for 30 percent to 40 percent of the 
teachers. The "other" professional development category included anything 
other than PRISM training and cluster leader training, and we do not know 
exactly which professional development opportunities teachers were reporting. 

The least influential factors were school staff (including district curriculum 
specialists, principal or assistant principal, school resource teacher or cluster 
leader, and other teachers) and specific professional development (including 
PRISM training and cluster leader training). Overall, only about 10 percent of 
teachers said that the district curriculum specialist and principal or assistant 
principal had a great deal of influence over the content of their teaching while 
slightly more than one-half of the teachers said they had little or no influence. 
Nineteen percent of respondents reported that "other teachers" had a great deal 
of influence over course content, while 36 percent said that other teachers had 
little or no influence over what they taught. Similarly, PRISM training and 
cluster leader training did not appear to have substantial influence for many 
teachers. Only about 11 percent of teachers said PRISM training had a great deal 
of influence over the content and style of their mathematics teaching, and only 
24 percent of teachers indicated that cluster leader training had a great deal of 
influence over their mathematics content and teaching strategy. 18 



18 However, these figures for cluster leader training and PRISM training may be misleading because 
these forms of support were not readily available to all teachers. For example, there was one 



There were only minor differences between fifth- and eighth-grade 
mathematics teachers in the influence of these factors. For example, in middle 
schools, a supportive professional environment was associated with higher 
gains. Specifically, a greater percentage of eighth-grade mathematics teachers in 
high-gain schools than in low-gain schools (42 percent compared to 0 percent) 
reported that school staff (including district curriculum specialists, school, 
principals, and other teachers) had a strong influence on their teaching of 
mathematics. In addition, a greater percent of teachers from high-gain schools 
than low-gain schools (29 percent compared to 5 percent) reported that the 
principal or assistant principal had a great deal of influence on the content of 
their curriculum. Similarly, the district curriculum specialist had a great deal of 
influence on 25 percent of the teachers in high-gain schools compared to none of 
the teachers in low-gain schools. 

Similarly, some curriculum materials were more influential among eighth- 
grade mathematics teachers in high-gain than low-gain schools. Sixty-five 
percent of eighth-grade mathematics teachers in high-gain schools said the 
NCTM Standards had a great deal of influence over content and teaching strategy 
compared to 37 percent of eighth-grade mathematics teachers in low-gain 
schools. Similarly, a greater percentage of eighth-grade mathematics teachers 
from high-gain schools than low-gain schools (45 vs. 16 percent) indicated that 
supplementary mathematics materials had a great deal of influence over their 
teaching strategies. 

Textbooks 

The alignment of textbooks with curriculum is important, because 
mathematics teachers rely on textbooks for a large portion of their assignments. 
About 10 percent of teachers draw almost all their mathematics assignments (76 
percent-100 percent) from their textbooks. Roughly one-third of teachers obtain 
more than one-half of their assignments (between 51 percent and 75 percent) 
from textbooks. Another one-third obtain between 26 percent and 50 percent of 
their assignments from. .textbooks,. -and -the remainder, 23 percent, use textbooks 
for less than one-quarter of their assignments. Teachers also rely on 



cluster leader for each 20 teachers. In rural parts of the state, cluster leaders could be an hour's 
drive away from the teachers they were supposed to support, and this distance might greatly 
diminish their effectiveness. Similarly, PRISM training was only available in some areas of the 



supplemental books and materials. About three-quarters of the teachers used 
supplemental materials at least once a week. 

The large majority of fifth- and eighth-grade teachers indicated that 
textbooks supported the mathematics curriculum well in the areas of number 
and computation, and Geometry and measurement. However, as Table 16 shows, 
the level of support dropped considerably in other areas of mathematics. For 
example, less than one-half of the teachers reported that their textbooks provided 
good support for their efforts to make connections between mathematics and 
other subjects. 19 

Fifth-grade textbooks were somewhat better aligned with most aspects of the 
curriculum than eighth-grade textbooks, according to the teachers. The one 
exception was in the area of Algebraic ideas. Only one-third of the fifth-grade 
teachers said their textbooks supported teaching algebraic ideas well compared to 
three-quarters of the eighth-grade teachers 



Table 16 

Textbook Support for Mathematics Content (Percent of Teachers Whose Textbook Supported 
Content Area "Moderately Well" or "Very Well") 





Fifth 


Eighth 


Content Area 


Grade 


Grade 


Number and computation 


90 


78 


Geometry and measurement 


80 


65 


Statistics and probability 


55 


52 


Algebraic ideas 


34 


77 


Connections between mathematics & other subjects 


48 


30 


Mathematical communication 


65 


38 



Fifth-grade teachers from high-gain schools thought better of their textbooks 
than fifth-grade teachers from low-gain schools. For example, 50 percent of fifth- 
grade teachers from high-gain -schools -said 'their textbooks -supported "algebraic 
ideas" moderately or very well, compared to only 22 percent of teachers from 
low-gain schools. Sixty-nine percent of fifth-grade teachers from high-gain 



19 These results are somewhat difficult to interpret since we do not know which textbooks different 
teachers are using. 



schools said their textbooks supported “mathematical communication" well, 
compared to only 41 percent of teachers from low-gain schools. Unfortunately, 
we cannot associate these rating with specific textbooks. 

School Support 

Teachers provided a generally positive picture regarding the support they 
are given by their schools and districts. For example, three-quarters of the 
teachers reported that all or most of the materials they need to teach 
mathematics are available at their school. Table 17 summarizes teachers' 



Table 17 



School Support for Change (Percent of Teachers Who Agreed "Somewhat" or 
Statement) 


"Strongly" 


with 




Fifth 


Eighth 


Statement 


Grade 


Grade 


The school administration actively supports teachers' efforts to improve 
mathematics instruction 


93 


92 


Teachers in this school frequently share ideas and materials to improve 
mathematics teaching 


78 


80 


My school or district provides many opportunities for teachers to learn new 
mathematics concepts and practices 


45 " 


53 


Teacher have enough opportunities to meet together to work on 
mathematics lessons 


25 ... 


17 


Teachers have enough time to plan new lessons and prepare new materials 
in mathematics* 


24 


18 


Teachers in this school regularly observe each other teaching classes as 
part of sharing and improving instructional strategies 


13 


22 



* Reversed 



responses to questions about school and district support. Almost all teachers (93 
percent) agreed that their school administration actively supports teacher's 
efforts to improve mathematics instruction. They also confirmed teachers in 
their school frequently share ideas and materials to improve mathematics 
teaching. However, such sharing must take place outside of class because few 
teachers regularly observe each other's lessons. Time seemed to be a limiting 
factor on instructional improvement. Few teachers thought there was adequate 
time to meet together or to plan new lessons. This may explain the earlier 



finding that other teachers did not greatly influence what was taught or how it 
was taught (see p. 40). These results did not vary greatly across grade level or for 
teachers in high-gain and low-gain schools. 

The availability of remedial and enrichment opportunities in mathematics 
has changed during the past three years, but there is no consistent pattern to this 
change. About one-third of the fifth-grade teachers reported an increase in the 
availability of remedial services and about one-third reported a decrease. The 
bulk of the teachers reported that the availability of enrichment services in 
mathematics had not changed during the past three years, although the increases 
out numbered the decreases 36 percent to 18 percent. In eighth grade, the balance 
was tipped toward increased availability of both remedial and enrichment 
services. 

In high-gain schools twice as many teachers reported increases in remedial 
and enrichment services as in low-gain schools. For remedial services, 46 percent 
of teachers in high-gain schools reported an increase compared to 21 percent of 
teachers in low-gain schools. The comparable figures for enrichment services 
were 49 percent compared to 21 percent. 

At least partially as a result of these support systems, most teachers thought 
their schools' mathematics programs served the needs of all students moderately 
well. Overall, more than one-half of teachers rated their school's mathematics 
program either as outstanding (9 percent) or good (48 percent) in this regard, and 
all but 6 percent said it was at least adequate. 

Assessment and Grading Practices 

The majority of teachers reported that they have increased the amount of 
time they spend assessing students' mathematics skills — 67 percent of teachers 
who have taught at least three years reported an increase in teacher-initiated 
classroom assessment. Teachers assigned individual, short-answer open- 
response tasks most frequently. Slightly more than one-half of the teachers gave 
such tasks at least-once a week, and- about- one-third of-the teachers -assigned them 
one or two times a month. Individual, open-response tasks requiring a longer 
answer or a table, chart or figure were assigned almost as often. Group tasks were 
the next most frequent type of assessment, and multiple-choice questions were 
used least often. Thirty-nine percent of fifth-grade teachers and 51 percent of 



eighth-grade mathematics teachers never assigned multiple choice questions. 
Over seventy-percent of both groups assigned them at most once a semester. 

Table 18 shows the frequency with which teachers use various types of 
problems to assess students' mathematics skills and the change in frequency 
during the past three years. As was the case for other teaching practices, 
traditional approaches dominated, but the greatest changes was an increase in 
standards-based approaches. Teachers in both grade five and grade eight reported 
using problems like those learned in class far more frequently than difficult, 
novel, or extended problems. However, they also reported that their use of the 
more difficult, novel, and extended problems had increased the most during the 
past three years. 



Table 18 

Use of Types of Mathematical Problems (Percent of Teachers Who Use Problem Type "Frequently" 
and Percent Who Increased Use During the Past Three Years) 





Fifth Grade 


Eighth Grade 




Frequent . 


3-year 


Frequent 


3-year 


Problem type 


Use 


Increase 


Use 


Increase 


Traditional approaches 










Computations similar to those learned in class 


82 


u 


77 


7 


Word problems similar to those learned in class 


80 


31 


64 


26 


Standards-based approaches 










Problems that are more difficult than those 


30 


45 


20 


30 


learned in class 










Novel, non-routine problems 


20 


65 


19 


38 


Extended investigations 


21 


57 


15 


50 



For the most part teachers in fifth and eighth grade responded similarly. 
The one exception to this pattern was the use of novel, non-routine problems. 
More fifth-grade teachers reported they had increased their use of these problems 
in assessment than eighth-grade teachers (65 percent vs. 38 percent). 

Similarly, teachers in high-gain and low-gain schools responded similarly to 
questions about assessment. There were four interesting exceptions to this 
pattern. Fifth-grade teachers in schools that had the smallest gains on the KIRIS 
mathematics assessment during the second biennium were more likely to say 



that they have increased the amount of time they spend on mathematics 
assessment than teachers in schools that showed the largest gains (70 percent 
compared to 48 percent). In eighth grade, mathematics teachers in high-gain 
schools were more likely to report assessing students "frequently" using 
"problems that are more difficult than those learned in class" than mathematics 
teachers in low-gain schools (40 percent compared to 10 percent). Multiple-choice 
tests were used infrequently in both grade levels, but teachers in high-gain 
schools were more likely to report that they never use multiple-choice questions 
to assess students in mathematics than teachers in low-gain schools (59 percent 
compared to 40 percent). Also in both grades more teachers in high-gain schools 
than low-gain schools increased their use of extended investigations to assess 
students in mathematics (62 percent compared to 41 percent). 

Teachers used a large number of factors in assigning course grades. The vast 
majority of the teachers gave the following factors: "a moderate amount" or "a 
great deal" of weight in grading: open-response exercises (95 percent); short 
quizzes or tests (94 percent); portfolio entries (85 percent); homework results (83 
percent); culminating projects (82 percent); homework timeliness (77 percent); 
performance tasks (74 percent); and participation in discussions (72 percent). It is 
interesting to note the factors that teachers counted "little or none" in grading: 
mid-year or final exams (73 percent); interest in and attitude toward mathematics 
(54 percent); and textbook chapter tests (41 percent). 

Many teachers have changed their grading practices over the past three 
years. The majority of teachers reported they had increased the amount they 
counted open-response exercises and portfolio entries in assigning grades (72 
percent and 57 percent respectively). The only factor whose importance declined 
for an appreciable number of teachers (23 percent) was textbook chapter tests. 

For the most part, there was little difference in grading practices between 
fifth and eighth-grade teachers. The only exceptions were the weight given to 
participation in discussion and interest and attitudes toward mathematics by the 
two groups of teachers. A higher percentage of fifth-grade teachers than eighth- 
grade teachers gave "a moderate amount" or "a great deal" of weight in course 
grades to participation in discussions (86 percent compared to 58 percent) and 
interest in and attitude toward mathematics (65 percent compare to 28 percent). 



There were a few interesting differences in grading practices between 
teachers in high-gain and low-gain schools. In both grades, more teachers in 
high-gain than low-gain schools increased the amount they counted 
performance tasks in grading (43 percent vs. 20 percent). In addition, in fifth 
grade, more teachers from high-gain schools than low-gain schools (42 percent 
compared to 18 percent) counted mid-year or final exams "a moderate amount" 
or "a great deal" in mathematics grading. Fewer teachers in high-gain schools 
than low-gain schools counted participation in discussion (93 percent compared 
to 70 percent) substantially in grading, but more teachers from high-gain than 
low-gain schools decreased the amount they counted mid-year or final exams (36 
percent compared to 17 percent) and homework results (19 percent compared to 0 
percent) in assigning grades. 

Portfolios 

Although mathematics portfolios were officially "on hiatus" during 1996-97 
and did not contribute to KIRIS scores, there appeared to be strong momentum 
on the part of districts and teachers to continue their use in both fifth and eighth 
grades. Three-quarters of the teachers continued to have students compile 
mathematics portfolios on either a formal or an informal basis. About one-half 
of these teachers reported they were required to do so by their school or district; 
the other half did it voluntarily. Furthermore, 44 percent of the fifth-grade 
teachers and 56 percent of the eighth-grade mathematics teachers said they 
intended to score the portfolios this year, although only about two-fifths of these 
teachers intended to score them formally using the old Kentucky rubric or a 
modified version of it. 

On the other hand, teachers had more negative than positive opinions 
regarding the use of portfolios in their classes. 20 Such results were not surprising. 
Weaknesses reported by classroom teachers contributed to the decision to place 
the mathematics portfolios on hiatus for two years while they were re-designed. 
Over 90 percent of the teachers agreed or strongly agreed that "the heavy 
emphasis on writing in the portfolios has caused students to become tired of 
writing." This may explain, in part, why over 80 percent disagreed with the 
statement that "my students enjoy doing portfolio tasks more than other 



20 It seems fair to assume that the decision to redesign the mathematics portfolio system sent a 
negative signal about the existing process. 



mathematics assignments." Over 90 percent of the teachers agreed that 
"portfolios make it difficult to cover the regular mathematics curriculum." 

Teachers' opinions were mixed regarding the importance of the 
mathematical skills emphasized by the portfolios. Over 80 percent agreed that 
portfolios "place too much emphasis on process skills (e.g., writing, 
mathematical communication) rather than content (e.g., operations, fractions)." 
But a similar percentage agreed that the mathematics portfolios "reflect aspects of 
mathematics that I believe are important." Unfortunately, the survey did not 
provide an opportunity for teachers to explain which aspects of mathematics 
were important. 

Portfolios had positive effects on instructional innovation, but they 
continued to be perceived as a burden by many teachers. Roughly two-thirds of 
the teachers in both grades agreed that portfolios "led me to be more innovative 
in planning mathematics lessons and activities." Yet, teachers were evenly 
divided about the demands of the portfolios. One-half of the teachers agreed and 
one-half disagreed that it "had become easier to find good portfolio tasks" and 
that "portfolios were less of a burden each year." 

Teachers were least enthusiastic about the use of portfolios for 
accountability. Two-thirds questioned the use of portfolio scores to judge 
students' overall mathematics achievement. They were more positive about the 
use of portfolios for classroom assessment. One-half of the teachers indicated that 
they use portfolio scores to help judge how effective their own teaching has been. 

Finally, it appears that teachers are developing a common understanding of 
the desired features of portfolio prompts. There was almost universal agreement 
that good tasks should be "interesting and engaging," "related to the students' 
real world," and "closely tied to [the] current mathematics lesson." There was 
some disagreement regarding the amount of structure that should be present in a 
good portfolio prompt. Three-quarters or more of the teachers in both grade 
levels agreed that tasks should be "open ended," but there was variation in how 
they interpreted this. Seventy-three percent of fifth-grade teachers thought 
portfolio prompts should have more than one correct solution, which seems 
consistent with being open-ended. However, 56 percent thought prompts should 
include step-by-step directions for students, which some would consider to be 
inconsistent with being open-ended. Fewer eighth-grade mathematics teachers 



agreed that good tasks should have multiple solutions (58 percent) or step-by-step 
directions (38 percent). 

In fifth grade, there were some differences in portfolio practices between 
teachers in high-gain and low-gain schools. Seventy-four percent of the fifth- 
grade teachers in high-gain schools whose students were compiling portfolios 
were doing so because their school or district required them to. In low-gain 
schools, only 37 percent of those who were compiling portfolios were required to 
do so. Although most fifth-grade teachers were critical of portfolios as measures 
of student achievement, the criticism were greater among teachers in low-gain 
schools. Sixty-eight percent of fifth-grade teachers in high-gain schools agreed 
that portfolios were a poor method of judging of students' overall mathematics 
achievement compared to 89 percent of fifth-grade teachers in low-gain schools. 

Since the mathematics portfolios are currently being redesigned, these 
results may be most useful as a baseline for comparing teacher beliefs and 
behaviors before and after the new portfolio system is in place. 

WRITING TEACHING AND LEARNING 
Teacher Education and Experience 

Fourth-grade teachers and seventh-grade writing teachers have strong 
educational backgrounds (albeit in a variety of subject areas), considerable 
teaching experience, and moderate lengths of service in their current schools. 21 
The majority of the fourth- and seventh-grade teachers surveyed reported that 
they have a master's degree or higher. In fourth grade, 59 percent of the teachers 
have a Masters' degree and 21 percent have a Kentucky Rank One Certificate, 
which requires 30 hours of coursework beyond a Masters' degree. In seventh 
grade, 53 percent of the writing , teachers have a Masters' degree and 22 percent 
have a Kentucky Rank One Certificate. 

The median number of years of teaching experience for fourth-grade 
teachers was nine years and for seventh-grade writing teachers it was seven 
years. Most of this teaching was done in their current school. The median 
number of years of teaching experience in their current school was eight years for 
fourth-grade teachers and five years for seventh-grade writing teachers. 

21 These results are similar to those for fifth-grade teachers and eighth-grade mathematics 



The majority of the fourth-grade teachers (71 percent) were certified in 
Elementary education (as were the majority of fifth-grade teachers). However, 
only 22 percent of fourth-grade teachers specialized in writing, English, or 
language arts in either their credential preparation program or when they 
renewed their teaching credentials. In contrast, seventh-grade writing teachers 
were more diverse in terms of certification and were far more likely to specialize 
in writing, English, or language arts. Their teaching certificates were divided 
roughly equally among Elementary education (39 percent), Middle School (46 
percent), and Secondary Grades (32 percent). 22 The majority of the seventh-grade 
writing teachers (58 percent) specialized in writing, English, or language arts in 
their teacher preparation program, and almost one-third (30 percent) specialized 
in writing, English, or language arts when they had their credentials renewed. 
Overall, about two- thirds of the seventh-grade writing teachers specialized in 
writing, English, or language arts during preservice training or credential 
renewal. 

There was no difference in teaching experience or subject matter 
preparation between teachers in high-gain and low-gain schools in either grade 
level. 



School and Classroom Organization 

Classroom organization for fourth-grade instruction was more varied than 
we anticipated. About half (48 percent) of Kentucky fourth-grade teachers 
described their school's instructional organization as consisting of self-contained 
classrooms but an equal number (47 percent) described it as “semi- 
departmentalized," with students receiving instruction in academic subjects 
from more than one teacher. 23 By the time students reach the seventh grade, they 
are typically assigned to a different teacher for each subject. In 82 percent of 
schools, seventh-grade instruction was completely departmentalized, while 15 
percent of schools had a semi-departmentalized seventh grade. We found no 
relationships between school and classroom organization and KIRIS gains in 
fourth or seventh grade. 



22 Teachers can hold teaching certificates in more than one category. 

23 See chapter on mathematics for a discussion of what semi-departmentalized classrooms might 
look like. 
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The majority of teachers reported that their schools did not group students 
by ability when assigning them to classrooms. However, ability grouping was 
more common at the seventh-grade level than at the fourth-grade level. 
Thirteen percent of seventh-grade teachers work in schools where students were 
assigned to classes by ability, compared to only four percent of fourth-grade 
teachers. Ability grouping at the school level was not associated with differences 
in KIRIS gain scores in writing at either grade level. 

Ability grouping can also occur within classes, but the majority of teachers at 
both grade levels reported that they seldom used ability grouping for writing 
instruction. Almost 60 percent of teachers indicated that they "rarely" group 
students by ability for writing instruction and another 18 percent only use such 
grouping occasionally. 24 There was not much difference between teachers in 
high-gain and low-gain schools in the use of ability grouping for writing' 
instruction. 

One of the reasons teachers may not use ability grouping for writing 
instruction is that students work individually during a large part of their writing 
lessons. On average, teachers reported that 42 percent of the instructional time in 
writing is spent with students working on their own, another 27 percent of the 
time is spent with students working in small (2-4 students) or large (5 or more 
students) groups. For the remaining one-third of the instructional time, teachers 
reported that they taught writing to the whole class. This pattern was similar for 
teachers in low-gain and high-gain schools. 

Professional Development and Preparedness 

Almost all fourth- and seventh-grade teachers participated regularly in 
professional development. Teachers reported that they benefited from the 
training they received, and expressed confidence in their ability to teach writing. 
There was little or no difference in professional development or preparedness 
between teachers in high-gain and low-gain schools at either grade level. 

All of the teachers surveyed participated in formal professional 
development activities in the current school year and almost all participated in 
each of the past two years as well (95 percent and 93 percent for 1995-96 and 1994- 
95, respectively). In fourth grade, the professional development experiences 



24 Similar results were reported for reading instruction; 54 percent of teachers rarely group students 
by ability for reading and 14 percent use such grouping occasionally. 



covered a wide range of topics, but Language Arts was very popular. Three- 
quarters of the teachers (79 percent) reported that much of their professional 
development ("a moderate amount" or "a great deal") focused on Language Arts 
curriculum and instruction. A majority of the teachers reported that much of 
their professional development focused on mathematics curriculum and 
instruction (53 percent) or curriculum and instruction in other subjects (56 
percent). In seventh grade, more than three-quarters of the seventh-grade 
writing teachers (78 percent) indicated that "a moderate amount" or "a great 
deal" of this professional development focused on Language Arts curriculum 
and instruction compared to only 21 percent for mathematics curriculum and 
instruction and 35 percent for curriculum and instruction in other subjects. 

KERA goals and academic expectations, preparation for the KIRIS 
assessments, and portfolio development and scoring were popular topics for 
professional development for teachers in both grade levels. Overall, more than 
70 percent of the teachers participated in professional development that focused 
"a moderate amount" or "a great deal" on portfolios development (85 percent), 
portfolio scoring (84 percent), preparing students for the KIRIS assessments (73 
percent), and KERA goals and academic expectations (72 percent). In contrast, 
fewer than one-half participated in professional development that focused 
strongly on classroom assessment methods (44 percent) or the use of educational 
technology (49 percent). 

Specific training in teaching writing was also widespread. Ninety-five 
percent of fourth-grade teachers and seventh-grade writing teachers reported that 
they had received special training to teach writing at some point in their careers. 
This included undergraduate courses, graduate course, continuing education, 
training through the Kentucky Writing Program, and other (non-specified) 
professional development. More than one-half of the teachers (59 percent) had 
received training through the Kentucky Writing Program at some point in their 
careers. In addition, during the last three years, 27 percent of the fourth-grade 
teachers and 39 percent of the seventh-grade writing teachers acted as trainers 
providing professional development in writing for other teachers. A greater 
percentage of fourth-grade teachers from high-gain than low-gain schools 
received special training to teach writing from "other professional development" 
(92 percent compared to 72 percent). Otherwise there was no difference in writing 
training between teachers in high-gain and low-gain schools. 



Teachers indicated that professional development had a positive effect on 
their ability to teach writing. The majority of teachers reported that professional 
development helped them "a moderate amount" or "a great deal" with every 
aspect of teaching writing that we asked about. Specifically, professional 
development helped teachers: improve students' writing portfolios (73 percent), 
teach prewriting skills (68 percent), integrate writing across the curriculum (66 
percent), teach students how to draft a piece (64 percent), teach writing through 
the study of literature (60 percent), teach students how to revise their writing (60 
percent), conference with students about their writing (59 percent), teach students 
how to edit their writing (57 percent), and teach students how to publish their 
writing (56 percent). There were no sizable differences between the responses of 
fourth-grade teachers and seventh-grade writing teachers. 

The majority of teachers in both grades felt "fairly well prepared" or "very 
well prepared" to teach most aspects of writing, and there was very little 
difference between the responses of teachers at the two grade levels (see Table 19). 



Table 19 



Teacher Preparation for Writing Topics (Percent of Teachers Who Were 
Well" Prepared) 


"Fairly Well 


" or "Very 


Topic 


Fourth 

Grade 


Seventh 

Grade 


Mechanics (spelling, punctuation, and capitalization) 


92 


91 


Sentence structure 


88 


89 


Logical organization 


87 


86 


Use of effective language 


86 


85 


Idea development 


85 


77 


Focused purpose 


81 


79 


Writing in a variety of genres/ forms 


78 


73 


Awareness of audience 


61 


74 


Tone /voice 


56 


67 



At both grade levels, the two topics that the greatest number of teachers felt well 
prepared to teach were "mechanics" and "sentence structure." The three topics 
that the fewest number of teachers felt well prepared to teach were tone /voice, 
awareness of audience, and writing for a variety of genres. Thus, teachers feel 



most comfortable teaching traditional aspects of writing rather than the reform- 
based aspects of writing. There were no associations between KIRIS gains in 
writing and teacher preparedness. 

Curriculum Emphasis 

Allocation of Instructional Time Across Subjects 25 

Fourth-grade teachers who taught all subjects reported that instructional 
time had stayed the same or increased in almost all academic subjects.26 
However, more teachers reported increased time in the subjects tested by KIRIS 
in their grade level than in other subjects. Table 20 shows the median hours per 
week devoted to each subject by fourth-grade teachers, and the percent of teachers 



Table 20 

Fourth-Grade Subject Coverage and Change in Coverage (Percent of Teachers 
Reporting Increase in Coverage During Past Three Years) 



Subject 


Median 
Hours per 
Week 


Percent 

Increasing 

Coverage 


Mathematics 


5 


34 


Writing* 


5 


90 


Arts and Humanities 


2 


44 


Practical Living/ Vocational Studies 


1 


30 


Science* 


4 


45 


Social Studies 


4 


15 


Reading* 


5 


36 



* Denotes subject tested in KIRIS in grade four; other subjects are tested in grade 
five. 



25 Thirty-two percent of fourth-grade teachers indicated that they did not teach all subjects and 
skipped the questions about the allocation of instructional time across subjects. They were excluded 
from the results reported in this section. 

26 Very few teachers reported decreasing the time they spent on any subject. This could occur if 
teachers increased the total amount of instructional time by reducing non-instructional activities, i f 
they conducted more inter-disciplinary activities that were counted as time for both subjects, or if 
their estimates were inflated. In any case, there is no reason to question the relative changes in 
time among subjects. 



who increased the time spent on each subject. Fourth-grade teachers reported 
spending the most time on reading, writing and mathematics. Furthermore, 
almost all of the fourth-grade teachers (90 percent) reported that they increased 
the amount of time they devoted to writing during the past three years. 

The allocation of time across subject was similar for fourth-grade teachers in 
high-gain schools and low-gain schools. However, more fourth-grade teachers in 
high-gain schools than in low-gain schools reported increases in the time spent 
on reading (54 vs. 29 percent), science (48 vs. 28 percent), arts and humanities (56 
vs. 32 percent)/ and practical living /vocational studies (39 vs. 11 percent). 

One of the goals of KERA was to encourage teachers to integrate writing 
with other subjects. Integration occurred quite often in fourth grade. 27 Over 
three-quarters of the fourth-grade teachers integrated writing with another 
subject at least once a week. Integration was most frequent in social studies and 
science, where about two-thirds of the teachers (66 percent and 63 percent, 
respectively) reported integrating writing with each of the two subjects at least 
once a week. There were no consistent differences between fourth-grade teachers 
in high-gain and low-gain schools in the frequency of integrating writing with 
other subjects. 

There were substantial increases in the amount of integration that occurred 
in fourth grade. Almost all fourth-grade teachers reported that they increased the 
amount that they integrated writing with at least one other subject over the past 
three years. The majority of teachers reported increasing the integration of 
writing with science (69 percent), social studies (59 percent), and mathematics (56 
percent). A greater percentage of fourth-grade teachers in high-gain than low- 
gains schools increased the integration of writing with social studies (74 percent 
vs. 45 percent) and with practical living/vocational studies (44 percent vs. 25 
percent). 

Seventh-grade teachers were asked a slightly different question: how often 
they worked on student writing with teachers from other subjects. Such inter- 
teacher collaboration about writing does not occur that frequently. Only 24 
percent of writing teachers reported working with the social studies teacher at 
least once a week and fewer than 15 percent of the writing teachers reported 

27 A variety of different activities may be reported under the heading "integration," ranging from 
the use of writing while teaching or assessing another subject to the teaching of writing in the 
context of another subject. 



working with teachers in any of the other subjects we asked about that 
frequently. However, there has been some increase during the past three years in 
the frequency with which seventh-grade writing teachers work with teachers in 
other subjects. Fifty-one percent of seventh-grade writing teachers reported that 
they have increased the frequency with which they work with teachers in at least 
one other subjects on student writing over the past three years. Social studies and 
science were the subjects where the most teachers reporting increasing the 
frequency of integration with writing (50 percent and 45 percent, respectively). 
More seventh-grade writing teachers from high-gain schools than low-gain 
school integrated writing with social studies at least once a week (38 vs. 9 
percent). In addition, more seventh-grade writing teachers from high-gain than 
low-gain schools increased the frequency with which they work with teachers 
from the subjects of mathematics(54 percent vs. 15 percent), science (70 percent 
vs. 48 percent), and arts and humanities (50 percent vs. 31 percent). 

Allocation of Instructional Time within Writing 

Teachers were asked about the frequency of coverage of nine different 
aspects of writing. Table 21 shows that over three-quarters of the teachers in both 



Table 21 



Frequency of Writing Coverage (Percent of Teachers Who Cover the Topic at 
Least Once a Week) 



Topic 


Fourth Grade 


Seventh Grade 


Mechanics 


93 


82 


Use of Effective Language 


87 


82 


Idea Development 


86 


80 


Sentence Structure 


84 


74 


Logical Organization 


82 


79 


Focused Purpose 


79 


81 


Awareness of Audience 


73 


80 


Tone/Voice 


65 


66 


Writing in a Variety of Genres 


55 


53 



grade levels cover all but two of the areas at least once a week. The topics that 
were addressed at least weekly by the largest percent of teachers were mechanics. 



use of effective language, and idea development. The two aspects of writing that 
were addressed at least weekly by the fewest number of teachers at both grade 
levels were tone /voice and writing in a variety of genres. 

Most teachers also reported increasing the frequency of coverage of most 
aspects of writing during the past three years. More than one-half of the teachers 
in both grades reporting increasing the time they devote to all aspects of writing 
except sentence structure and mechanics (see Table 22). More teachers reported 
increasing time spent on awareness of audience and focused purpose than on 
any of the other aspects of writing, which is consistent with the emphasis of the 
Kentucky reform of writing. 



Table 22 

Change in Writing Coverage (Percent of Teachers Who Increased Coverage 
During the Past Three Years) 



Topic 


Fourth Grade 


Seventh Grade 


Mechanics 


38 


45 


Use of Effective Language 


56 


52 

•J. 


Idea Development 


66 


65 


Sentence Structure 


53 


45 


Logical Organization 


59 


64 


Focused Purpose 


71 


72 


Awareness of Audience 


70 


76 


Tone/Voice 


61 


66 


Writing in a Variety of Genres 


58 


56 



There were no differences in the frequency of coverage between teachers in 
high-gain and low-gain schools. However, more teachers in high-gain schools 
than low-gain schools reported increasing their coverage of writing in a variety 
of genres/forms during the past three years (68 percent vs. 46 percent in fourth 
grade and 77 percent vs. 48 percent in seventh grade). In addition, more seventh- 
grade writing teachers in high-gain schools than low-gain schools reported 
increasing the frequency of coverage of use of effective language (74 percent vs. 
54 percent), sentence structure (69 percent vs. 40 percent), logical organization (80 
percent vs. 59 percent), tone/voice (78 percent vs. 59 percent), mechanics (62 
percent vs. 40 percent) and idea development (78 percent vs. 59 percent). 



Instructional Practices 



We examined a number of aspects of writing instruction, including the 
teachers' overall philosophy and instructional approach, expectations for student 
performance, specific teaching strategies, and student learning activities. 

Philosophy and Approach to Teaching Writing 

Teachers characterized their overall approaches to teaching writing on four 
dimensions defined by contrasting statements. For three of the four dimensions, 
the statement at one end of the continuum was more consistent with a 
standards-based approach and the statement at the other end was more 
consistent with a traditional approach. 28 These three dimensions concerned goals 
for student writing, the order of presentation of the elements of writing, and the 
teachers' overall goals for instruction. Teachers indicated whether they 
emphasized the mechanics of writing vs. communicating clearly, taught spelling 
and grammar before purpose and audience vs. teaching purpose and audience 
before or simultaneously with spelling and grammar, and emphasized 
transmitting knowledge to students vs. encouraging thinking and questioning. 
Each dimension had five levels, including one that was midway between the two 
descriptors. Teachers marked one of five points that reflected their approach. 

On the three dimensions that related to writing, two-thirds or more of the 
teachers rated themselves closer to the standards-based end of the continuum 
than to the traditional end. For example, 82 percent of fourth-grade teachers and 
77 percent of seventh-grade writing teachers placed themselves closer to the goal 
"to help students communicate clearly through writing" than the goal "to help 
students learn the mechanics of writing." Similarly, 68 percent of fourth-grade 
teachers and 84 percent of seventh-grade writing teachers identified more closely 
with teaching purpose and audience before (or while) teaching spelling and 
grammar than with teaching spelling and grammar before purpose and 
audience. Over three-quarters of both groups identified their overall goals more 
closely with encouraging thinking and questioning than with transmitting 
knowledge. There were no significant differences in overall approach to teaching 
writing between teachers in high-gain and low-gain schools. 



28 The fourth dimension concerned overall philosophy of instruction (in-depth study of a few topics 
vs. comprehensive coverage of many topics) which was not strongly associated with either 
approach to teaching writing. Teachers tended to place themselves in the middle on this 
dimension. 



Responses to additional questions about teaching approach demonstrated 
some uncertainty on the part of teachers about the amount of structure that is 
needed to promote effective writing. Almost all teachers agreed that students 
require a good deal of practice to become competent writers and that different 
students learn to write in different ways. However, as in mathematics, there was 
less agreement about the degree to which students should be left to puzzle things 
out for themselves. Table 23 shows that at least a small majority of teachers 
agreed with three somewhat contradictory statements. On the one hand, most 
teachers believed students should be given step-by-step instructions and should 
have their errors correctly quickly. On the other hand, a small majority endorsed 
the notion that students should be allowed to figure things out for themselves, 
which would not be the case if the first prescriptions were strictly followed. There 
were strong similarities between the responses of fourth and seventh-grade 
teachers, and between teachers in high-gain and low-gain schools. 



Table 23 

Teachers' Opinions About Learning to Write (Percent of Teachers Who Agreed with Statement) 





Fourth Grade 


Seventh Grade 


Statement 


Some- 

what 

Agree 


Strongly 

Agree 


Some- 

what 

Agree 


Strongly 

Agree 


Students' errors should be corrected quickly so they 
do not become frustrated 


45 


13 


61 


13 


Students should receive step by step directions to 
aid in producing a successful, final, written product 


52 


32 


50 


33 


Students learn best if they have to figure things out 
for themselves instead of being told or shown what 
to do 


38 


13 


41 


10 



Expectations for Student Performance 

A key assumption of the Kentucky educational reform is that all students 
can learn to high levels, but teachers of writing find it difficult to endorse this 
belief in practice (see Table 24). Although almost two-thirds of fourth-grade 
teachers and one-half of seventh-grade writing teachers believe all student can 
become successful writers, less than one-half of fourth-grade teachers and one- 
quarter of seventh-grade writing teachers think all students can attain existing 



grade level expectations. In fact, a high percentage of teachers in both grade levels 
agreed that there are some students who will never perform above the level of 
Novice in writing. Consistent with these beliefs, only about one-half of the 
teachers thought the same standards should be used to evaluate the work of all 
students. In most cases, teachers in low-gain and high-gain schools held similar 
expectations. The one exception to this pattern was that a lower percentage of 
fourth-grade teachers in high-gain schools than low-gain schools thought all 
students could learn fourth-grade writing by the end of fourth grade (37 percent 
compared with 55 percent). 

Teachers' responses suggest it may be difficult to set appropriately high 
expectations that motivate greater achievement without over-burdening 
students. Over 80 percent of teachers agreed that teachers in their school push 
students fairly hard in academic subjects, but over 80 percent also agreed that if 
you put too much pressure on students they can lose their enthusiasm for 
writing. 



Table 24 

Teachers' Expectations for Students' Writing (Percent of Teachers Who Agreed "Somewhat" or 
"Strongly" with Statement) 





Fourth 


Seventh 


Statement 


Grade 


Grade 


With appropriate practice and instruction, all students can become 
successful writers 


65 


48 


There are some students who will never perform above the level of 
Novice, no matter how much effort the teacher devotes to them 


67 


81 


I believe all students can learn fourth [seventh] grade writing by the end 
of fourth [seventh] grade 


47 


22 


Teachers should use the same standards in evaluating the work of all 
students in class 


53 


58 



Interestingly, teachers think it is more difficult to improve the performance 
of students who are writing well than students who are writing poorly. A greater 
percentage of teachers thought it would be more difficult to move students from 
Proficient to Distinguished in writing than from Apprentice to Proficient or from 
Novice to Apprentice. At both grade levels, almost one-half of the teachers 
thought it would be most difficult to take students who were Proficient writers 



and help them become Distinguished writers. About one-quarter thought it 
would be most difficult to move students from Apprentice to Proficient, and 
only 14 percent thought that moving Novice writers to Apprentice would be the 
most difficult. 

Teaching Strategies 

There has been considerable change in emphasis within writing at both 
grade levels. Fifty-one percent of all teachers reported that the content of their 
writing lessons had changed a major amount during the past three years. 29 An 
additional 30 percent indicated that the content had changed a moderate amount. 
These changes are reflected in teachers' overall approach to writing and the way 
they address specific aspects of writing instruction, including the purposes for 
which student write, the way written pieces are started, and the kinds of feedback 
teachers give to students. These topics will be discussed in the succeeding 
paragraphs. 

The writing process approach to instruction was used frequently by more 
teachers than any other method. Ninety-three percent of teachers at both grade 
levels use writing process instruction frequently. This compared to 67 percent 
who frequently used integrated reading and writing, 46 percent who frequently 
used writing to learn, and 44 percent who frequently used mechanics or skills- 
based instruction. Fourth-grade teachers were more likely than seventh-grade 
writing teachers to use writing to learn (53 percent vs. 39 percent) and mechanics 
or skills-based approaches (54 percent vs. 34 percent) frequently. There was no 
difference in approach between fourth-grade teachers in high-gain and low-gain 
schools, although seventh-grade writing teachers in high-gain schools were 
much more likely to use skill-based instruction frequently than were seventh- 
grade writing teachers in low-gain schools (66 percent vs. 20 percent). While the 
writing process approach is the most . common, many teachers regularly use 
multiple methods for teaching writing. 

Teachers ask students to write for a number of different purposes, with 
fourth-grade teachers assigning writing for a greater variety of purposes than 
seventh-grade writing teachers (see Table 25). The largest percentage of teachers 
in both grade levels gave writing assignments in which students were asked to 
share facts or information or to write for a purpose of their own choosing. The 



29 Eleven percent had not taught writing long enough to answer the question. 



least common purposes at the fourth-grade level were pieces designed to reflect 
on student's own growth and pieces written to persuade an audience. Least 
common at the seventh-grade level were pieces designed to tell a story and pieces 
written to persuade an audience. This is consistent with seventh-grade writing 
teachers' responses to a question about writing styles. They reported that, on 
average, 36 percent of students' writing is narrative, 25 percent is descriptive, 20 
percent is expository and 19 percent is persuasive. 

There were few differences in writing purpose between teachers from high- 
gain and low-gain schools. In fourth grade, more of the writing assignments 
from teachers in high-gain schools ask the student to "tell a story" at least once a 
week (46 compared to 21 percent). In seventh grade, teachers from high-gain 
schools appear to be somewhat more flexible about assignments. A greater 
number of teachers from high-gain schools than low-gain schools gave weekly 
assignments in which the students wrote for a purpose of his or her own 
choosing (48 percent vs. 30 percent). 



Table 25 

Purposes for Writing (Percent of Teachers Assigning Writing for Each Purpose Once or Twice a 
Semester or Less and Once a Week or More) 





Fourth Grade 


Seventh Grade 


Purpose 


Rarely 


Weekly 


Rarely 


Weekly 


Share facts or information 


12 


51 


34 


32 


A purpose of the student's own choosing 


17 


46 


31 


35 


Tell a story 


16 


37 


55 


10 


Explain a process or concept 


26 


32 


48 


19 


Analyze or evaluation a situation, person, place 
or thing 


32 


30 


49 


21 


Reflect on student's own learning or growth 


44 


30 


46 


22 


Describe a scene 


26 


28 


43 


23 


Evoke a feeling or mood 


- - 28 - 


- - 28 


35 


28 


Persuade an audience 


33 


20 


56 


11 



Teachers used a variety of starting points for student writing on a regular 
basis, and their choices are consistent with the standards-based approach to 



writing, fable 26 shows the percentage of teachers who used various starting 
points at least once a week. The most common starting point for writing at both 

Table 26 

Starting Points for Student Writing (Percent of Teacher Who Used Strategy at Least Once a Week 
and Percent Who Increased Use During Past Three Years) 

Fourth Grade Seventh Grade 



Weekly 3-year Weekly 3-year 
Starting Point Use Increase Use Increase 



Traditional approaches 


A story starter 


17 


12 


6 


22 


A topic assigned by the teacher 


50 


21 


24 


25 


A book read by the student 


23 


31 


18 


30 


. Standards-based approaches 


An idea or experience chosen by the student 


59 


42 


55 


62 


A model piece of writing 


36 


46 


40 


58 


An experience shared by students in the class 


37 


31 


28 


35 


Work done in another subject area 


45 


57 


10 


35 



grade levels was an idea or experience chosen by the student, which was used on 
a weekly basis by 56 percent of the teachers. Moreover, almost one-half of the 
teachers increased the use of student selected writing topics over the past three 
years. Teachers also increased the use of model pieces of writing as a starting 
point for student writing. The least common starting point for writing at both 
grade levels was a “story starter," which was used on a weekly basis by 11 percent 
of the teachers across the two grade levels. In fact, the two strategies whose use 
was reduced during the past three years by a substantial percentage of teachers 
were story starters (17 percent) and topics assigned by teachers (23 percent). The 
most notable difference between the grade levels was the use of work done in 
another subject area. As one might expect, seventh-grade writing teachers drew 
upon work from another subject only infrequently, although one-third had 
increased the frequency of this practice during the past three years. There were no 
major differences in starting points for student writing between teachers in high- 
gain and low-gain schools. 



Fourth-grade students wrote more often than seventh-grade students, and 
short written pieces were far more common than long written pieces at both 
grade levels. Two-thirds of the fourth-grade teachers reported that their students 
wrote short pieces (one to two paragraphs in length) almost daily compared to 42 
percent of seventh-grade writing teachers. (Of course, seventh-grade students 
may be producing written work in other classes, which is not included in these 
reports.) Two-thirds of the fourth-grade teachers and slightly less than one-half 
of the seventh-grade writing teachers reported that students produced middle- 
length pieces (one to two pages) on a weekly basis. About one-half of the teachers 
in both grade levels reported that students produced long pieces (three or more 
pages in length) on a monthly basis. However, one-third of the teachers at both 
grade levels, said that students produced long written pieces only once or twice 
each semester. There were no differences in the reported frequency of short and 
middle-length student writing between teachers in high-gain schools and low- 
gain schools. However, a greater number of teachers in high-gain schools than 
low-gain schools reported at least monthly production of written pieces at least 
three pages in length (67 percent vs. 51 percent among fourth-grade teachers; 68 
percent vs. 53 percent among seventh-grade writing teachers). 

Teachers have many opportunities to give students feedback on their 
writing during the year. This feedback can take the form of written or oral 
comments about various aspects of the piece. More than one-half of the teachers 
reported that they almost always commented on the following aspects of 
students' writing: the strengths of the piece, how the student could improve the 
piece, focused purpose, idea development, logical organization, and mechanics. 
Between 40 percent and 50 percent of the teachers said they almost always 
commented on awareness of audience, sentence structure, and use of effective 
language. The vast majority of the teachers (70 percent to 90 percent) reported 
that they commented on these nine factors in more than half of the written 
pieces they reviewed. The two features that were least often reflected in teachers' 
feedback were tone/voice and sensitivity to genre/ form. Between 21 percent and 
25 percent of the teachers mentioned ' these aspects of writing on almost all the 
pieces of written work they reviewed. There was no difference in feedback on 
written work between teachers in fourth and seventh grade, or of between 
teachers in high-gain and low-gain schools. 



At both grade levels, teachers were asked to imagine that they had just 
finished a lesson on persuasive writing and were having a conference with a 
student who had turned in a written draft. Figures 1 and 2 show the passages 
provided to teachers in fourth and seventh grade, respectively. 



Dear Mom and Dad, 

I think you should take me to the Olympics because I would have fun. It would not cost lots of money. 
We could stay in a hotel and eat in restrants. I would like to see the kayaking. Dad said he would 
take me kayaking when I get older. I would learn alot watching the lympic kayakers, 
your son. Josh 

Figure 1. Sample Fourth-Grade Writing Passage 



Yesterday I was woken up around five-thirty in the morning by sound of a motor running outside my 
window. I looked out my window and saw that it was the neighbor removing the snow in his drive 
using a snow blower. I laid back down and tried to return to sleep but could not. I began to ponder the 
question: Does this man have the right to run a snow blower this early in the morning? 

I think that there should be a city law outlawing the use of snowblowers between 10 pm and 6 am 
because they are noisy and cause people to lose sleep. Also, they are not necessary since it is possible 
to remove the snow without making such a ruckus. I realize that some people have long driveways 
and this is difficult but that was there choice and others should not have to lose sleep over it. 

In conclusion, since some people do not seem willing to be neighborly, I think city action is required 
to solve this problem. 



Figure 2. Sample Seventh-Grade Writing Passage 



Although the passages differed, teachers in both grade levels were asked 
about the same set of dimensions. There was consensus among the teachers 
about which features would be important to emphasize in a student conference. 
In fact, more than half the teachers said they would emphasize each of the 
features (see Table 27). Almost all of the teachers said they would place a 
moderate or great deal of emphasis on persuasiveness (99 percent), evidence to 
support claims, and idea development (both 97 percent). One difference between 
the grades was in regard to sentence structure— more fourth-grade teachers 
would emphasize sentence structure than would seventh-grade teachers (82 
percent compared to 61 percent). Teachers from low- and high-gain schools only 
differed on one dimension: more fourth-grade teachers from high-gain schools 
would place a moderate or great deal of emphasis on "use of language" (81 
percent vs. 62 percent). 



Table 27 



Teacher Feedback on Student Writing (Percent of Teachers Placing a Moderate 
Amount or a Great Deal of Emphasis on Each Element) 



Element 


Fourth Grade 


Seventh Grade 


Evidence to support claims 


98 


96 


Mechanical errors 


61 


57 


Idea development 


95 


99 


Persuasiveness 


99 


97 


Sentence structure 


82 


61 


Voice 


76 


80 


Organization 


90 


84 


Use of language 


71 


63 


Awareness of audience 


82 


90 



Student Learning Activities 

We also asked teachers about the types of activities students engage in when 
learning to write. Teachers ask students to do a variety of activities during 
writing instruction, including activities that emphasize the mechanics of 
writing, activities that emphasize the process of writing, and activities that 
emphasize the KIRIS scoring guides (see Table 28). Teachers have increased the 
frequency with which students do all the activities, with the greatest increases in 
the frequency of conferences and discussions of the writing portfolio scoring 
guides. A greater percentage of fourth-grade teachers than seventh-grade teachers 
reported weekly practice on mechanics, weekly conferences with individual 
students, and weekly critiques of writing with the whole class. A greater 
percentage of seventh-grade writing teachers in high-gain schools than low-gain 
schools reported weekly student practice on the mechanics of written English (92 
percent vs. 61 percent). Similarly, greater percentages of seventh-grade writing 
teachers in high-gain schools than low-gain schools reported increases in the 
frequency of -individual student conferences - about writing (84 percent vs. 68 
percent), critiquing writing with the whole class (76 percent vs. 57 percent), 
reading and discussing a professional authors work (58 percent vs. 35 percent), 
and learning about genres/ forms (55 percent vs. 33 percent). In fourth grade, 
teachers in high-gain schools were more likely than teachers in low-gain schools 



to report increases in individual conferences about writing (73 percent vs. 53 
percent) and learning about genres/ forms (55 percent vs. 30 percent). 

Students spend considerable time working on various aspects of their 
written work. In a typical two-week period teachers at both grade levels reported 
that students spend an average of about four hours on drafting; an average three 
hours each on conferencing, prewriting, revising, editing, and publishing; and 
about two hours on studying examples of effective writing. 



Table 28 

Student Writing Activities (Percent of Teachers Reporting Activity Occurs at Least Once a Week 
and Percent Who Increased the Frequency During the Past Three Years) 





Fourth Grade 


Seventh Grade 


Activity 


Weekly 

Use 


3-year 

Increase 


Weekly 

Use 


3-year 

Increase 


Students practice the mechanics of written English 


85 


31 


63 


40 


Conferences with individual students about their 
writing 


88 


63 


60 


77 


Class discussion of the Kentucky writing portfolio 
scoring guide 


49 


58 


39 


. 67 


Students learn about a variety of genres/ forms 


40 


44 


32 


48 


Critique a piece of student writing with the whole 
class 


40 


49 


19 


63 


Students use the Kentucky scoring guide to assess 
their own or other students' writing 


34 


50 


36 


63 


Students read and discuss a professional author's 
writing 


29 


32 


27 


46 



Although all pieces of writing may not be addressed in the same way, a 
substantial proportion of teachers said that students engaged in multiple 
activities on almost all of the pieces they wrote. Table 29 shows the percentage of 
teachers reporting that students engaged in each writing activity on almost all 
the pieces they wrote. Revision and editing are the most prevalent activities. 
Students are less likely to gather information or conduct research prior to 
writing. Peer conferencing is more prevalent in seventh grade than in fourth 
grade. Although students do not publish every piece they write,- 76 percent of the 
teachers reported that students publish more than half the pieces they write. 



There were no differences in student activities among fourth-grade teachers in 
high-gain and low-gain schools. However, in seventh grade a greater percentage 
of writing teachers in high-gain than low-gain schools reported that students 
almost always engage in making notes or an outline (55 percent vs. 36 percent) 
and revising the piece at least once (82 percent vs. 63 percent). 



Table 29 

Activities of Student Writing (Percent of Teachers Indicating Students Engaged in the Activity for 
" Almost All" Written Pieces) 



Activity 


Fourth 

Grade 


Seventh 

Grade 


Gather information /conduct research before they write 


2 


6 


Make notes or an outline before they write 


34 


52 


Define the purpose and audience 


43 


58 


Use conferencing with peers to improve their writing 


41 


63 


Use conferencing with the teacher to improve their writing 


56 


49 


Revise the piece at least once 


70 


68 


Edit the piece to correct errors in mechanics 


70 


68 


Publish the piece for others to read 


38 


36 



Influences on Practice and Support for Change 

KIRIS 

The majority of teachers reported that the KIRIS writing portfolios have had 
a positive impact on writing instruction: 84 percent of fourth-grade teachers and 
71 percent of seventh-grade writing teachers reported that the writing portfolios 
had either a small or large positive impact. Surprisingly, seventh-grade writing 
teachers in low-gain schools were more positive about the impact of the writing 
portfolios than their colleagues at high-gain schools (89 percent vs. 62 percent). 
This difference did not occur among fourth-grade teachers in high-gain and low- 
gain schools. 

Although most fourth-grade teachers and seventh-grade writing teachers 
reported that KIRIS writing portfolios have had a positive impact on writing 
instruction, they attributed score improvements on KIRIS more to other factors 
(see Table 30). Differences between groups of students was the factor cited the by 



largest percentage of teachers in both grades as contributing "a great deal" to score 
gains. The next most often cited factor was increased student familiarity with the 
KIRIS portfolio assessment. On the positive side, two- thirds of the teachers in 
both grade levels reported that improvements in students' writing skills made at 
least a "moderate" contribution to score gains. 30 



Table 30 

Reasons for KIRIS Score Gains in Writing (Percent of Teachers Who Reported that Each Factor 
Contributed "A Great Deal" to KIRIS Score Gains in their School) 



Factor 


Grade 4 


Grade 7 


Increased student motivation 


16 


2 


Differences between groups of students from one biennium to 
the next 


33 


46 


Broad improvements in students' writing skills 


13 


8 


Specific improvements in the writing skills assessed by KIRIS 


12 


15 


Increased student familiarity with the KIRIS portfolio 
assessment 


32 


22 



State and Local Support for Reform 

Writing teachers have had access to a wealth of information and support to 
enhance writing instruction. KDE and other sources have developed curriculum 
materials, provided training relating to portfolio development and scoring, and 
offered a wide range of professional development opportunities. We asked about 
the influence of five broad types of resources: curriculum guides and textbooks, 
assessments, local school and district staff, formal professional development, and 
personal experience and beliefs. 31 

The most influential item on the list was the KIRIS writing portfolios. Over 
90 percent of the teachers indicated that the writing portfolios had a great deal of 



30 Sixty-nine percent of fourth-grade teachers and 67 percent of seventh-grade writing teachers said 
"broad improvement in students' writing skills" contributed at least a moderate amount to writing 
score gains. Fifty-one percent (fourth grade) and 57 percent (seventh grade) said "specific 
improvements in the writing skills assessed by KIRIS" contributed at least a moderate amount to 
writing score gains. One KDE staff person indicated that there are no differences between broad 
writing skills and the skills being assessed in KIRIS, which may have confused teachers. 

31 We asked separately about the influence of these items on what writing teachers teach and how 
they teach writing, In general, teachers responded similarly to these two sets of questions, so we 
have chosen to present only the results for factors that influence what they teach. 



influence on what they taught in writing. There were six other resources that 
had a great deal of influence on the content of writing instruction for a majority 
of the teachers. Three of the influential resources were curriculum materials — 
the Kentucky Writing Portfolio Teacher Handbook (65 percent). Core Content for 
Writing Assessment (KDE, undated) (72 percent), and Transformations (KDE, 
1995a) (56 percent). Two of the most influential resources were professional 
development activities — portfolio scoring training (57 percent) and portfolio 
development training (54 percent). The last item that strongly influenced writing 
instruction for a majority of teachers was personal experience and beliefs (62 
percent). 

Six other items exerted some influence for the vast majority of teachers. 
These were district curriculum guides, supplemental writing materials, cluster 
leaders, other teachers, the Kentucky Writing Program, and "other" professional 
development. The least influential factors were district curriculum specialists, 
principal or assistant principal, textbooks, commercial standardized tests, district 
or school tests, and university writing courses. 

There were only minor differences between fourth-grade teachers and 
seventh-grade writing teachers or between teachers in high-gain schools and 
low-gain school in the reported influence of these factors. The most notable 
grade-related difference concerned how influential teachers found the portfolio 
scoring training: 68 percent of the fourth-grade teachers reported that it had a 
great deal of influence on what they taught whereas only 45 percent of seventh- 
grade teachers responded that way. The only sizable differences between teachers 
in high-gain schools and low-gain schools concerned the Kentucky Writing 
Program and portfolio development training, and the differences were in 
opposite directions. Teacher in low-gain schools were more likely than teachers 
in high-gain schools to report that the Kentucky Writing Program had a great 
deal of influence on what they taught in writing (52 percent and 34 percent in 
fourth grade, respectively; 40 percent and 24 percent in seventh grade, 
respectively). Conversely, teachers in high-gain schools were more likely than 
teachers in low-gain schools to report that portfolio development training had a 
great deal of influence on what they taught in writing (54 percent vs. 37 percent 
in fourth grade, and 56 percent vs. 43 percent in seventh grade). 



Textbooks 

Although the majority of teachers use a language arts or writing textbook 
(70 percent), they do not rely on it for writing assignments. Seventy- three percent 
of teachers who use a language arts or writing textbook reported that fewer than 
25 percent of their writing assignments come from the textbook. Teachers rely 
more heavily on supplemental books and materials. Sixty-one percent of the 
teachers used supplemental materials at least once a week and about a quarter (26 
percent) use them almost daily. Teachers in fourth and seventh grade reported 
similar textbook use as did teachers in high-gain and low-gain schools. 

School Support 

Teachers in both grade levels provided a generally positive picture 
regarding the support they are given by their schools and districts. For example, 
about three-quarters (74 percent) of the teachers reported that all or most of the 
materials they need to teach writing effectively are available at their school. Table 
31 summarizes teachers' responses to questions about school and district support. 



Table 31 

School Support for Change (Percent of Teachers Who Agreed "Somewhat" or "Strongly" with 
Statement) 



Statement 


Fourth 

Grade 


Seventh 

Grade 


The school administration actively supports teachers' efforts to improve 
writing instruction 


89 


90 


Teachers in this school frequently share ideas and materials to improve 
the teaching of writing 


83 


86 


My school or district provides adequate opportunities to learn new writing 
concepts and practices 


64 


70 


Teachers have enough time to plan new lessons and prepare new materials 
for writing* 


24 


28 


Teacher have enough opportunities to meet together to work on writing 
lessons 


25 


25 


Teachers in this school regularly observe each other teaching classes as 
part of sharing and improving instructional strategies 


22 


26 



* Reversed 



Most teachers agreed that their school administration actively supports teacher's 
efforts to improve writing instruction. They also confirmed that teachers in their 
school frequently share ideas and materials to improve the teaching of writing. 
However, time seemed to be a limiting factor on instructional improvement. 
Few teachers thought there was adequate time to plan new lessons and prepare 
new materials. Also, few teachers reported that they have enough time to meet 
with other teachers to work on writing lessons or to observe each other's lessons. 
These results did not vary greatly across grade levels or for teachers in high-gain 
and low-gain schools. 

In many schools, the availability of remedial and enrichment opportunities 
in writing has changed during the past three years, but there is no consistent 
pattern to this change. In fourth grade, more teachers reported an increase in the 
availability of remedial services in writing than reported a decrease (42 percent 
and 19 percent, respectively). The same was true for enrichment services (45 
percent and 12 percent, respectively). In seventh grade, teachers were more 
evenly divided in reporting increases and decreases in both remedial services (31 
percent and 29 percent, respectively) and enrichment services (35 percent and 19 
percent, respectively). Surprisingly there were no differences regarding remedial 
and enrichment support between teachers in high-gain and low-gain schools. 

Overall, most teachers thought their schools' writing program served the 
needs of all students at least moderately well. More than half of teachers rated 
their school's writing program either as outstanding (14 percent) or good (41 
percent) in this regard, and all but 13 percent said it was at least adequate. In 
fourth grade, more teachers in high-gain schools than low-gain schools rated 
their school's writing program as at least good (61 percent vs. 39 percent). There 
were no such differences in the seventh-grade results. 

Assessment and Grading Practices 

Teachers consider a number of different aspects of writing when assigning 
grades to student written pieces. Teachers in grades four and seven ranked the 
factors in roughly the same order in terms of importance in grading, but the 
percent of seventh-grade teachers for whom a particular aspect of a piece was 
"very important" in assigning a grade was higher than the percent of fourth- 
grade teachers responding to the same aspect. 



The four factors that most fourth-grade teachers indicated to be very 
important in grading were idea development (80 percent), accomplishing the 
focused purpose of the task (78 percent), logical organization (71 percent), and the 
quality and creativity of the ideas (73 percent). About one-half of the fourth-grade 
teachers also said awareness of audience (56 percent) and appropriate tone and 
voice (44 percent) were very important. Far fewer fourth-grade teachers reported 
that the following were "very important:" length (8 percent), mechanics (25 
percent), appropriate sentence structure (33 percent), and use of effective 
language (38 percent). 

Most seventh-grade writing teachers indicated that idea development (91 
percent), logical organization (89 percent), accomplishing the focused purpose of 
the task (87 percent), and the quality and creativity of the ideas (71 percent) were 
all very important in assigning grades to written work. In addition, most 
seventh-grade teachers marked awareness of audience (71 percent) as very 
important. Fewer teachers, but still a majority, marked that appropriate tone and 
voice (59 percent) and use of effective language (57 percent) were very important 
in grading. The factors judged to be important by the fewest seventh-grade 
teachers were length (7 percent), mechanics (29 percent), and appropriate 
sentence structure (40 percent). 

For the most part, there was little difference in grading practices between 
teachers in high-gain and low-gain schools. The two exceptions were the 
importance given to use of effective language and appropriate tone and voice. In 
seventh grade, more teachers in high-gain than low-gain schools reported that 
use of effective language was very important in assigning grades (61 percent vs. 
39 percent respectively). Also, more teachers in high-gain than low-gain schools 
reported that appropriate tone and voice was very important in assigning grades 
(69 percent vs. 50 percent respectively). The same trends were evident to a lesser 
extent in fourth grade. 

Almost all of the teachers in both grades (87 percent in fourth grade and 88 
percent in seventh grade) reported that their students receive a report card grade 
in language arts. About one-half said their students also receive a grade in 
writing. In fourth grade, over 90 percent of the teachers said their students 
receive a report card grade in reading, as well. In seventh grade, two- thirds of the 
teachers said there was a reading grade assigned to students. The language arts 
grade reflected reading, writing, speaking and listening, with the bulk of the 



emphasis divided evenly between reading and writing. In fourth grade, speaking 
and listening contribute only 5 percent each to the language arts grade; in 
seventh grade they contribute only 10 percent each to the language arts grade. 

Portfolios 

As noted above, the KIRIS writing portfolios have had a strong influence on 
writing instruction. This was reflected in teachers' opinions about the influence 
of portfolios and the demands they place on teachers. Two-thirds or more of the 
teachers agreed that students are writing more and better as a result of portfolios 
(80 percent), that portfolios have led them to be more innovative in planning 
writing lessons and activities (78 percent) and that portfolios have caused them 
to de-emphasize mechanics (69 percent). Teachers also agreed that portfolios 
have led them to allow students to make more decisions about their own writing 
(78 percent). 

On the other hand, teachers report that portfolios are burdensome. About 
three-quarters of the teachers agreed that portfolios make it difficult to cover the 
regular curriculum (79 percent) and that the heavy emphasis on the writing 
portfolios has caused students to become tired of writing (73 percent). In addition, 
the burden does not seem to be decreasing for most teachers. Forty-nine percent 
of fourth-grade teachers and 73 percent of seventh-grade teachers disagreed with 
the statement "Portfolios are less of a burden this year than last." Scoring is a 
substantial part of the burden felt by teachers. Overall, 80 percent of teachers 
agreed that portfolio scoring is too time consuming. Although many teachers 
continue to see the portfolios as burdensome, they did report that it has become 
easier to find good ideas for portfolio tasks (80 percent). 

There were very few differences in opinion between fourth-grade teachers 
and seventh-grade writing teachers. However, there were some differences 
between seventh-grade teachers in high-gain and low-gain schools. A greater 
percentage of seventh-grade writing teachers from low-gain schools than high- 
gain schools agreed that students are writing more and better as a result of 
writing portfolios (86 percent vs. 66 percent) and that writing portfolio scores are 
a good measure of [their] students' overall writing achievement (61 percent vs. 34 
percent). 

Because writing portfolio scores are the main measure of writing proficiency 
in KIRIS, it is important to examine the conditions under which students 



produce their portfolio pieces. There was considerable variation in teachers' 
reports of the origin of writing prompts and the amount of revision that was 
done to assessment portfolio pieces, but there was greater similarity in the level 
of feedback teachers gave to students and the process of choosing final portfolio 
entries. 

Some teachers supplied almost all the prompts for student portfolio pieces 
while other teachers supply almost none of the prompts. About one-fifth of 
teachers (21 percent) reported that most student portfolio writing begins with a 
prompt supplied by the teacher. About half of teachers (52 percent) reported that 
student portfolio writing begins with a teacher-generated prompt one-quarter to 
one-half of the time. The remaining teachers, 27 percent, reported that student 
portfolio writing almost never begins with a prompt. 

Most pieces in students' assessment portfolios were revised two or more 
times before they are completed. On average, about 20 percent of assessment 
portfolio entries were revised once, about 55 percent were revised two or three 
times, and about 35 percent were revised four or more times. However, there 
was considerable variation among teachers in these percentages. For example, in 
fourth grade the percent of pieces revised only once ranged from zero to 100 
percent. In the typical fourth-grade classroom only 10 percent of the pieces were 
revised only once, but in about one-fifth of the classrooms 20 percent or more of 
the pieces were revised one time. These differences become clearer at the other 
end of the spectrum. In the typical fourth-grade classroom, only 10 percent of the 
pieces are revised four or more times. However, in about one- fifth of the fourth- 
grade classrooms, 50 percent or more of the pieces were revised four or more 
times. Similar variations in revision practices were reported by seventh-grade 
writing teachers. 

There was comparable variation among teachers in the amount of time 
students worked on their portfolio pieces. On average, fourth-grade teachers 
reported that ten percent of assessment portfolio entries were completed 
(including revisions) in one to one and - one-half hours, 18 percent were 
completed in one and one-half to two hours, and 68 percent were completed in 
more than two hours. Again, there were large differences among teachers. For 
example, in the typical fourth-grade classroom, students spent more than two 
hours each on 80 percent of their pieces, but in about one-fifth of the classrooms 
students spent more than two hours on only 25 percent of their pieces. The 



results were similar in the seventh grade. On average, seventh-grade teachers 
reported that three percent of assessment portfolio pieces were completed in less 
than one hour, eight percent were completed in one to two hours, 24 percent in 
two and one-half to four hours, 19 percent in four and a half to six hours, and 45 
percent in more than six hours. 32 There were variations among teachers, as well. 

In seventh grade, teachers in high-gain schools reported that their students 
spent more time completing assessment portfolio entries than teachers in low- 
gain schools. For example, the typical seventh-grade writing teacher in a high- 
gain school reported that 50 percent of assessment portfolio entries are worked 
on for more than six hours each compared to 20 percent of assessment portfolio 
entries in low-gain schools. 

Almost all students receive frequent feedback from teachers as they work on 
their portfolio entries and they also receive feedback from other individuals. 
Nearly all teachers in both grades (98 percent) reported that their students 
frequently or always receive feedback from the teacher while drafting and 
revising assessment portfolio entries. Feedback from other sources varied by 
grade level. Most fourth-grade teachers reported that students frequently 
received feedback on their assessment portfolio entries from other students (65 
percent) and from parents or other adults at school (41 percent). Few teachers (6 
percent) reported that students received frequent feedback from parents or other 
adults outside of the school. In seventh grade, almost all teachers (87 percent) 
reported that students frequently received feedback on their assessment portfolio 
entries from other students. However, far fewer reported frequent feedback from 
parents or other adults, either at school (31 percent) or outside of the school (20 
percent). 

The large majority of teachers (87 percent) reported that assessment 
portfolio entries were selected from a broad collection of students' written work 
accumulated over the course of the year. Ninety-five percent of seventh-grade 
teachers and 73 percent of fourth-grade teachers allow students to choose writing 
pieces from previous years for -revision -and inclusion in their assessment 
portfolio. Fourth-grade teachers exercise more influence on the choice of pieces 
that go into students' assessment portfolios than seventh-grade writing teachers. 
On average, fourth-grade teachers reported that about one-third of the 



32 The fourth-grade and seventh-grade surveys had different response options on this question, so 
the results cannot be compared directly. 



assessment portfolio entries are chosen by the student alone, about one-half are 
chosen by the student with input from the teacher, and the remainder are chosen 
by the teacher with some input from the students. In seventh grade, about one- 
half of the entries are chosen by the student alone, about 40 percent by the 
student with input from the teacher, and on only about 10 percent does the 
teacher have substantial influence. 

Addressing the scoring criteria and demonstrating growth in performance 
are paramount concerns in choosing assessment portfolio pieces. Ninety-five 
percent of teachers indicated that it was either "somewhat important" or "very 
important" that pieces demonstrate achievement on all scoring criteria. Over 
ninety percent of teachers also said it was important that pieces show growth in 
performance. Two-thirds of the teachers also said the following criteria were 
important in selecting pieces for the assessment portfolios: neat and polished in 
appearance (82 percent), exemplary in one or two scoring dimensions (72 percent) 
and similar to best examples from scoring training (67 percent). More seventh- 
grade teachers in high-gain schools than in low-gain schools indicated that it was 
important for student's work to be exemplary in one or two scoring dimensions 
(80 percent vs. 62 percent). 

Overall, about half the teachers believe that the writing portfolios are good 
measures of students writing. About half agreed (52 percent) and half disagreed 
(48 percent) with the statement "Writing portfolio scores are a good measure of 
my students' overall writing achievement." 

SUMMARY AND CONCLUSIONS 

Survey responses leave little doubt that KERA and KIRIS have had effects 
on mathematics and writing curriculum and instruction in Kentucky. Although 
Kentucky mathematics and writing teachers continue to cover much of the 
traditional content and employ traditional teaching methods, they are increasing 
their use of standards-based approaches in both subjects. These changes are 
consistent with the direction signaled by KIRIS, particularly the portfolios, and 
they are supported by state curriculum materials and professional development 
used by the teachers. The mathematics reform represents a greater break with 
past practices than the writing reform, and this can be seen in differences in the 
mathematics and writing surveys and teachers' responses to them. Within 
subjects, there are relatively few differences between elementary and middle 



schools, and those that exist primarily reflect differences in the organization of 
the schools, e.g., subject matter specialization. More importantly, despite the 
alignment between the standards, KIRIS assessments, state curriculum materials 
and training opportunities, there are few associations between KIRIS gains and 
reported classroom practices. 

Teacher Preparation and Classroom Organization 

Overall, Kentucky fourth- and fifth-grade teachers, seventh-grade writing 
teachers and eighth-grade mathematics teachers had high levels of formal 
professional training and moderate levels of teaching experience and tenure in 
their present positions. The majority of teachers at all four grade levels had a 
masters' degree or higher, and about one-quarter have Kentucky Rank One 
Certification, which requires 30 hours of coursework beyond a masters' degree. 
Most fourth- and fifth-grade teachers specialized in elementary education (rather 
than a specific subject) during their college training or their credential renewal 
process. In contrast, two-thirds of seventh-grade writing teachers and three- 
quarters of eighth-grade mathematics teachers specialized in their subject area 
during their initial training or credential renewal. However, this also means that 
one-third of seventh-grade writing teachers and one-quarter of eighth-grade 
mathematics teachers in Kentucky did not specialize in the subject we asked 
about at any point in their formal training. 

In addition to their formal training, all Kentucky teachers in the grades and 
subjects we studied availed themselves of professional development 
opportunities on an annual basis. They chose professional development that 
focuses on KERA goals and outcomes, curriculum in their subject, and the KIRIS 
assessment in their subjects. Teachers rated professional development highly, 
particularly the support it provided for subject-matter instruction and the KIRIS 
assessments. Overall, teachers felt well prepared to teach most aspects of the 
subjects that we addressed. In fact, 20 percent to 30 percent of the teachers acted as 
trainers for other teachers. 

We found it surprising that a substantial number of fourth- and fifth-grade 
teachers did not teach in "self-contained" classes but worked in "semi- 
departmentalized" schools, where students were taught their academic subjects 
by more than one teacher. Team teaching would be an example of such an 



arrangement. In contrast, instruction in most middle schools was fully 
departmentalized, with teachers responsible for a single subject. 

Classroom Practices: Subject Matter Differences 

The mathematics and writing teachers we surveyed provided similar 
responses to questions about classroom practices. Both groups mixed traditional 
and standards-based curriculum and instruction, with increasing attention to the 
standards-based practices. Both aligned themselves philosophically with 
standards-based approach. They agreed that the KIRIS assessments and the 
curriculum materials provided by the state were the most potent influences on 
instruction in mathematics and writing, respectively. There were greater 
differences in their assessment practices, due in part to the fact that the 
mathematics portfolios were on hiatus when the survey was conducted. 

Curriculum and Instruction 

Writing, reading and mathematics receive the most coverage from teachers 
in fourth and fifth grades who were responsible for all subjects. In addition, 
teachers were integrating writing and mathematics more with other subjects. 
They appeared to be shifting their content emphasis to reflect the subjects that are 
tested at their grade level. For example, two-thirds of the fifth-grade teachers who 
taught for the past three years, increased the time they spend on mathematics, 
and many fifth-grade teachers also increased the time they spend on arts and 
humanities and practical living/vocational studies, subjects which are tested at 
the fifth-grade level. Most of the fourth-grade teachers increased the time they 
devote to writing during the past three years, and many increased their coverage 
of science and reading. Increased coverage was accompanied by regular 
integration of lessons across subjects, e.g., writing with science and social studies, 
mathematics with writing and science. 

Greater integration occurred at middle school, as well. Although we did not 
obtain strictly comparable information on this question from middle schools 
teachers, we do know that one-quarter of the seventh-grade writing teachers 
regularly worked with teachers in other subjects on student writing, and one-half 
increased the amount they work with teachers in other subject areas during the 
past three years. Similarly, one-quarter of the eighth-grade mathematics teachers 
regularly worked with teachers in other subjects on students' mathematics, and, 



during the past three years, 37 percent of eighth grade mathematics teacher 
increased the amount the work with teachers in other subjects. 

In both subjects, teachers continued to emphasize many of the traditional 
topics even while they were increasing coverage of reform topics. Within the 
field of mathematics, the traditional topics of numbers and computation 
received at least weekly coverage from almost all teachers in fifth and eighth 
grade. Far fewer teachers in either grade level covered less-traditional topics of 
geometry and measurement or statistics and probability on a weekly basis. 
However, most teachers in both grade levels increased their coverage of the non- 
traditional mathematics topics during the past three years. In writing, about 
three-quarters of the teachers at both grade levels addressed most aspects of 
writing on a regular basis. This included (in order of prevalence) mechanics, use 
of effective language, idea development, sentence structure, logical organization, 
focus purpose and awareness of audience. Tone/voice and writing in a variety of 
genres were addressed weekly by more than one-half of seventh-grade writing 
teachers. The topics for which the greatest number of teachers increased coverage 
were focused purpose and awareness of audience, both aspects of writing that are 
strongly associated with the writing reform. 

Teachers in both subjects aligned themselves philosophically with 
standards-based approaches to teaching more than traditional approaches. 
Moreover, teachers used both standards-based and traditional instructional 
strategies in their teaching, with increases occurring in the use of strategies 
consistent with reform. In mathematics, traditional instructional strategies were 
somewhat more common than standards-based strategies in the examples we 
posed (graphing, correcting student errors, and beginning and ending a new 
unit). However, most teachers endorsed student-centered responses over 
teacher-centered responses when dealing with individual student errors. In 
addition, a majority of teachers increased their use of standards-based 
instructional strategies during the past three years. Similarly, students were 
asked to engage in traditional learning activities more frequently than standards- 
based activities, but the frequency of the latter was increasing. 

In writing, teachers' instructional philosophies were more aligned with 
standards-based approaches to teaching writing than with traditional approaches, 
and writing instruction had changed in the direction of the reform during the 
past three years. For example, almost all teachers used the writing process 



approach, but most used other approaches, as well. Teachers asked students to 
write for many different purposes, and often let the students choose the purpose 
for the piece. Pieces most often began with an idea or experience chosen by the 
student, and this practices increased during the past three years. Students 
devoted considerable time to producing written work, and their most common 
activities were practicing the mechanics of written English and having 
individual conferences about their writing. 

Teachers in both subjects endorsed higher expectations for students — a 
prime tenant of the KERA reform — but have doubts about achieving it in 
practice. Although most mathematics teachers believed that all students can 
achieve more in mathematics than they have in the past, far fewer believed that 
all students can achieve at grade level . 33 Similarly, in writing, a majority of 
teachers believed all students can become successful writers, but only about half 
as many believed all students can achieve existing grade level expectations in 
writing. 

Support for Changing Practices 

In mathematics and writing most teachers were influenced greatly by the 
state curriculum materials and the KIRIS assessments. Of lesser importance were 
professional development, textbooks, and peers at the school level. Teachers in 
both subjects said they had the materials they needed to support instruction, and 
felt they received support and encouragement from the school principal and 
other teachers. However, few teachers in either subject thought they had 
adequate time to meet with their peers to plan new lessons. 

"Specifically, most mathematics teachers indicated that curriculum 
documents and materials developed by KDE and the National Council of 
Teachers of Mathematics had a great deal of influence on the content of their 
mathematics lessons. The KIRIS assessments themselves were also deemed quite 
influential by large numbers of teachers, particularly the mathematics portfolios 
and the open-response items. Only about one-third of the teachers regarded 
district curriculum guides, supplemental materials, and professional 
development as greatly influential. Many mathematics teachers drew substantial 
portions of their lessons from textbooks, and they reported that their texts 



33 Teachers would have responded in this way if they thought even a single student would not meet 
grade level expectations. 



supported traditional content better than other areas of mathematics, particularly 
connections between mathematics and other subjects. They also received support 
from their school administrators for their efforts to improve mathematics 
instruction. 

In writing, the KIRIS writing portfolios were the most potent factor in 
changing writing instruction. Almost all teachers indicated that the portfolios 
greatly influenced their teaching. Teachers also credited the portfolios with 
causing students to write more and better and for causing teachers to be more 
innovative in planning lessons. Furthermore, four of the six other factors that 
were perceived as influential by the majority of teachers were directly related to 
the portfolios: Kentucky Writing Portfolio Teacher Handbook, Core Content for 
Writing Assessment, portfolio scoring training and portfolio development 
training. Transformations and personal experience also influenced the content of 
most teachers' writing instruction. In contrast, textbooks were not very 
influential in terms of writing. Although the majority of teachers used a 
language arts or writing textbook they did not rely on it for writing assignments. 
Supplemental materials were used far more regularly in writing than textbooks. 

Teachers were less encouraging about the factors that affected students' 
scores on KIRIS. In mathematics, although many teachers thought that the KIRIS 
assessments influenced classroom practices, they did not think that school effects 
were primarily responsible for KIRIS gains. The vast majority of teachers in 
schools whose mathematics scores improved during the second biennium 
attributed those improvements to factors outside their influence. In particular, 
gains were attributed to difference between cohorts of students far more often 
than to improvements in specific KIRIS-related skills or improvement in 
general mathematical knowledge and skill. 

Similar responses were obtained in writing. The factor that largest 
percentage of teachers thought contributed greatly to KIRIS score gains was 
differences between cohorts of students. This was true despite the impact that 
KIRIS had on teaching practices and the positive-support teachers received for 
improving writing instruction. However, most teachers also thought 
improvements in students' writing skills made a moderate contribution to score 
gains. 



Assessment Practices 

Classroom assessment occurred frequently in both subjects, and the 
assessments were becoming more like KIRIS in format. Teachers continued to 
have mixed feeling about the KIRIS portfolios, which were seen to have a 
positive influence on instruction but to be overly burdensome. Specifically, in 
mathematics, teachers increased the amount of time they spent assessing 
students' mathematical skills, and they used problems that resembled KIRIS — 
short-answer open-response tasks and long-answer open-response tasks. Rarely 
did they use multiple-choice tests. When it came to problem solving, teachers 
used traditional types of problems more often than non-routine problems. 
However, during the past three years, most teachers increased their use of novel, 
non-routine problems and extended investigations that were consistent with 
Kentucky's stand ards-based approach. Although most teachers included many 
factors when assigning mathematics grades to students, during the past three 
years, most teachers have increased the weight that they give in student grades to 
open-response exercises and portfolio entries. 

In writing, the four factors that were very important to most teachers when 
grading individual written work were idea development, accomplishing the 
focused purpose of the task, logical organization, and the quality and creativity of 
the ideas. Fewer teachers said that length, mechanics, or sentence structure were 
very important in reviewing work. This is consistent with the standards for 
written communications. 

Teachers recognized strengths and weaknesses in the portfolio assessments, 
with writing teachers expressing more positive reactions than mathematics 
teachers. 34 Concerning the mathematics portfolios, teachers gave negative 
comments about the heavy emphasis on writing and the demands the portfolios 
placed on classroom time that made it difficult to cover the curriculum. Teachers 
agreed that portfolios had a positive effect on innovation, but they remained a 
burden on teacher and student time. Despite the burden, most teachers asked 
students to compile mathematics portfolios during 1996-97 even though the 
portfolios were not required by KDE. In writing, teachers were more enthusiastic. 
They agreed that the portfolios had many positive effects on teaching practices 
and student writing. However, they also found the writing portfolios to be 

34 The mathematics portfolios were being revised by the state at the time of this study, and that 
may have signaled to teachers that there were problems with the system. 



burdensome and said the portfolios made it difficult to cover the regular 
curriculum. 



Classroom Practices: Grade Level Differences 

There were a few interesting differences between classroom practices in 
mathematics and writing in elementary schools and middle schools. Most 
differences were related to the content of elementary and middle school courses 
and the organization of elementary and middle schools. 

In mathematics, the responses from fifth-grade teachers and eighth-grade 
mathematics teachers were quite similar with respect to their philosophies about 
teaching mathematics, expectations for students, curriculum emphases, 
instructional practices, assessment practices, and views about support for 
mathematics reform. There were noteworthy differences in background. Fifth- 
grade teachers had more teaching experience than eighth-grade mathematics 
teachers, and a higher percentage had master's degrees. However, eighth-grade 
mathematics teachers had more specialized training in mathematics, and, 
possibly as a result, a greater percentage felt adequately prepared to teach statistics 
and probability and algebraic ideas. 

In addition, there a number of differences in how the two groups of teachers 
taught mathematics, some of which appear to be consistent with differences in 
the way elementary and middle schools are organized. Fifth-grade teachers were 
more likely than eighth-grade teachers to do the following: ask open-ended 
questions and show connections between mathematics and other subjects; have 
students regularly write about mathematics and work on extended mathematics 
activities; begin a mathematics unit in non- traditional ways, such as drawing on< 
examples from other domains. In contrast, eighth-grade mathematics teachers 
are more likely than fifth-grade teachers to have students use calculators 
regularly to solve exercises and to cover the topic of algebraic ideas on a regular 
basis. 

There were -some differences in terms of classroom assessments, as well. A 
greater number of fifth-grade teachers than eighth-grade mathematics teachers 
had increased their use of novel, non-routing problems for assessing students in 
mathematics, and a greater number give substantial weight to participation in 
discussion and to interest and attitude toward mathematics when assigning 
grade. 
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In writing, similarities outweigh differences when comparing teaching in 
fourth and seventh grade, and many of the differences that do exist are related to 
the way instruction is organized at the two grade levels. For example, ability 
grouping is more common in assigning students to classroom in seventh grade 
than in fourth grade. Similarly, although the education level and teaching 
experience of fourth-grade teachers and seventh-grade writing teachers were 
similar, a higher percentage of seventh-grade writing teachers than fourth-grade 
teachers specialized in writing, English or language arts in the professional 
training program. 

There were some differences in emphasis in the way writing was taught in 
fourth and seventh grades, but most are related to the age of students and their 
stage of writing development. For example, more fourth-grade teachers had their 
students write short pieces (one to two paragraphs in length) on a daily basis, and 
middle middle-length pieces (one to two pages) on a weekly basis. More fourth- 
grade teachers than seventh-grade writing teachers frequently used writing to 
learn and skills-based approaches to teaching writing. Fourth-grade teachers 
asked students to write for a greater number of purposes on a regular basis than 
seventh-grade writing teachers. In addition, a higher percentage of fourth-grade 
teachers regularly drew upon work from other subjects as a basis for writing, 
although a higher percentage of seventh-grade writing teachers had increased the 
frequency of this practice during the past three years. A greater percentage of 
fourth-grade teachers than seventh-grade teachers reported weekly practice on 
mechanics, weekly conferences with individual students, and weekly critiques of 
writing with the whole class. Feedback from adults at school was more common 
in fourth grade than in seventh grade, while peer conferences were more 
prevalent in seventh grade. 

Classroom Practices: Relationship to KIRIS Gains 

We found no consistent differences between teaching practices in high-gain 
and low-gain schools based on second biennium KIRIS gains. There were many 
cases in which standards-based practices were associated with high gains in one 
subject or one grade level, but not in all cases. There were also some instances in 
which traditional practices were associated with high gains. Overall, the results 
are provocative, but not conclusive. 
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In mathematics, with a few notable exceptions, teachers in high-gain schools 
and low-gain schools have similar backgrounds and report similar beliefs and 
practices. The exceptions include some practices that would be characterized as 
standards-based and others that would be identified as traditional. Some of the 
factors distinguish high-gain schools from low-gain schools at both grade levels 
and others apply only to fifth grade or only to eighth grade. In terms of 
background, the only difference associated with KIRIS gains is self-reported 
preparation; a greater percentage of teachers in high-gain than low-gain schools 
feel well-prepared to teach the more advanced mathematics topics, including 
algebraic ideas and statistics and probability. 

Some curriculum and instructional differences are also associated with 
KIRIS gains, although most were detected only in eighth grade. More eighth- 
grade mathematics teachers in high-gain schools than low-gain schools did each 
of the following: increased their coverage of all mathematics topics during the 
past three years (with the greatest difference occurring in the areas of statistics 
and probability); had students practice computation on a daily basis; had students 
use calculators on a daily basis; give tests as culminating activities for 
instructional units. There was one difference in fifth grade in the other direction; 
a higher percentage of teachers in low-gain schools than high-gain schools 
increased coverage of statistics and probability during the past three years. In both 
grades, a greater percentage of teachers in high-gain schools than low-gain 
schools covered algebraic ideas on a regular basis. 

There were a few strong associations between assessment practices and 
KIRIS gains. Multiple-choice tests were used infrequently in both grade levels, 
but more teachers in high-gain schools than low-gain schools eschewed their use 
entirely. Instead, more teachers in high-gain schools increased their use of 
extended investigations to assess student performance, and also increased the 
amount they counted performance tasks in grading. In eighth grade, a greater 
percentage of mathematics teachers in high-gain schools than low-gain schools 
gave weekly tests or quizzes, and a greater percentage assessed students 
frequently using problems that were more difficult than those learned in class. In 
fifth grade, there were some opposite results. More teachers in low-gain schools 
than high-gain schools increased the amount of time they spend on mathematics 
assessment. 



Among eighth-grade mathematics teachers there were also some 
associations between KIRIS gains and school support for mathematics reform. A 
greater percentage of eighth-grade mathematics teachers in high-gain schools 
than low-gain schools said the content of their mathematics lessons was strongly 
influenced by other school staff, including the principal, resource teachers and 
other teachers. District curriculum specialists also had a strong influence on 
content among eighth-grade mathematics teachers in high-gain schools. 
Similarly, more eighth-grade teachers in high-gain schools said the NCTM 
standards had strong influence over mathematics content and practices, and that 
supplementary materials had a strong' influence on teaching practices. Not 
surprisingly, more eighth-grade mathematics teachers in high-gain schools than 
low-gain schools thought KIRIS had a positive impact on their schools' 
mathematics program. In fifth grade, more teachers in high-gain schools than in 
low-gain schools have textbooks that support the curriculum well. In both 
grades, twice as many teachers in high-gain schools than low-gain schools 
reported increases in the availability of remedial and enrichment support for 
students in mathematics. 

Similarly, in writing, there was little difference in background, teaching 
experience, or writing training between teachers in high-gain and low-gain 
schools, but there were of differences in other areas. On balance, the practices 
associated with KIRIS gains were consistent with the Kentucky writing reform, 
including integration of writing with other subjects, greater attention to certain 
aspects of writing such as writing in a number of genres/ forms, student-centered 
instruction, regular writing of longer pieces, and using more elements of the 
writing process, including drafting, conferencing and revising. For example, in 
both grades more teachers in high-gain schools than in low-gain schools 
integrated writing with other disciplines, particularly social studies. In addition, 
more seventh-grade writing teachers from high-gain schools than low-gain 
schools increased the frequency with which they work with teachers from other 
subjects. 

There were differences between high-gain and low-gain schools in the 
emphasis teachers placed on various aspects of writing. More teachers in high- 
gain schools increased their emphasis on writing in a variety of genres /forms 
during the past three years. In seventh grade, teachers in high-gain schools also 
increased their coverage of use of effective language, sentence structure, logical 
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organization, tone/voice, mechanics, and idea development. At the same time, a 
greater percentage of seventh-grade writing teachers in high-gain schools than 
low-gain schools used skill-based instruction regularly and had students 
regularly practice the mechanics of written English. 

There were some differences in the process of writing between high-gain 
and low-gain schools. In both grades, greater percentages of teachers in high-gain 
schools increased the frequency of individual student conferences about writing, 
and learning about genres /forms. In seventh grade, a greater number of teachers 
from high-gain schools had students regularly write for a purpose of their own 
choosing, and greater numbers of teachers also have students regularly engage in 
making notes or an outline and in revising written pieces at least once. In 
addition, more seventh-grade writing teachers in high-gain school increased the 
frequency of critiquing writing with the whole class and of reading and 
discussing a professional author's work. 

In addition, there were some differences in length of written work and in 
use of time. In both grades, a greater number of teachers in high-gain schools 
than low-gain schools reported at least monthly production of written pieces 
three pages in length or longer. In seventh grade, teachers in high-gain schools 
reported that their students spent more time completing assessment portfolio 
entries than teachers in low-gain schools. 

There were also some associations that were harder to explain. Teachers in 
low-gain schools were more positive about the impact of writing portfolios than 
teachers in high-gain schools. Similarly, teachers in low-gain were more likely to 
say that the writing portfolios had a positive effect on student writing, that 
portfolio scores were a good measure of writing achievement, that students 
could achieve grade level expectations in writing. Teachers in low-gain schools 
were more likely than teachers in high-gain schools to report that the Kentucky 
Writing Program had a great deal of influence on what they taught in writing. 

Conclusions 

There is ample evidence that teachers have changed classroom practices in 
response to KERA and KIRIS, and that KIRIS, in particular, has had a major 
influence on instructional behaviors. In general, this was true in mathematics 
and writing and in both the elementary and the middle school grades. Notably, 
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• Although teachers continued to cover traditional content and use 
traditional methods, they increased the frequency of standards-based 
approaches. 

• The majority of teachers 7 beliefs about teaching were closer to the 
standards-based approach than the traditional one. 

• Elementary teachers increased their coverage the subjects tested by KIRIS 
at their grade level. 

• Writing teachers perceived the portfolios to be a strong positive 
influence on students' writing ability. 

However, there were no consistent associations between specific teaching 
practices and biennial KIRIS gains. Some standards-based practices were more 
prevalent in high-gain schools, e.g., extended investigations in mathematics, 
longer-pieces and more frequent student conferences in writing. However, gains 
were also associated with some traditional practices, such as regular mathematics 
tests and regular practice on the mechanics of written English. In some cases, 
gains were associated with particular types of support, e.g., the influence of local 
school staff on mathematics curriculum. Overall, we did not find convincing 
evidence that a particular set of actions or policies would produce higher scores. 
If there is such a pattern it would appear to include both standards-based and 
traditional approaches. 

It is possible to draw both optimistic and pessimistic interpretations of these 
findings. On the one hand, standards-based reform is a relatively recent event in 
Kentucky. The first KIRIS examinations were just administered in 1991. The bulk 
of the literature on school reform indicates that it is difficult and time- 
consuming to change teaching practice. The surveys reveal widespread changes 
in the direction encouraged by KERA, which should be applauded. On the other 
hand, it is disappointing not to have found stronger links between practices and 
student performance. There are a number of reasons we may have failed to 
detect relationships that are really there, including the volatility of gain scores, 
the sensitivity of our instrument's, and the timing of our survey. 35 However, it 
may also be the case that Kentucky educators have not found a practice or 



35 We compared practices that occurred in 1996-97 and changes in practice during the preceding 
three-year period with school-level gain scores representing growth from 1992-94 to 1994-96. 



constellation of practices that consistently promotes higher achievement across 
groups of students. 

It is important to remember that this is only the first year of our study. A 
new survey was administered in the spring of 1998. This survey will provide 
information about practices in mathematics and writing in adjacent grade levels 
(e.g., fourth- and fifth-grade mathematics) so it should reveal more about the 
effects of standards-based assessment on classroom practices in grades that are not 
subject to accountability testing. Using subsequent years' KIRIS scores, we will 
also be able to look at associations between teachers' behaviors and student 
outcomes that are closer in time. In addition, our case studies of exemplary 
teaching will illuminate more about a highly selected set of teachers and the way 
the elements of their school context interact to promote the desired teacher and 
student behaviors. 
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